Re: USB stack

On Mon, Jan 8, 2018 at 3:20 PM, Mark Millard <markmi_at_dsl-only.net> wrote:

> [The involvement of sysutils/fusefs-simple-mtpfs
> and sysutils/fusefs-libs and multimedia/libmtp
> in order to use the Media Transfer Protocol (MTP)
> against the LG v30 likely explains much of the
> performance difference vs. local UFS file
> systems. I provide some related notes.]
>
> blubee blubeeme gurenchan at gmail.com wrote on
> Mon Jan 8 05:17:24 UTC 2018 :
>
> [Note: the original was in a reply to a Jon Brawn
> post. I've merged it back with my post.]
>
> > On 2018-Jan-7, at 11:46 AM, Mark Millard <markmi at dsl-only.net> wrote:
> >
> >> On 2018-Jan-7, at 10:23 AM, blubee blubeeme <gurenchan at gmail.com>
> wrote:
> >>
> >>
> >>
> >>> On Mon, Jan 8, 2018 at 1:41 AM, Mark Millard <markmi at dsl-only.net>
> wrote:
> >>>
> >>>> On 2018-Jan-7, at 7:50 AM, blubee blubeeme <gurenchan at gmail.com>
> wrote:
> >>>>
> >>>>> I ran this test and here's some results.
> >>>>> gstat -pd images:
> >>>>>
> >>>>> 18GB file from laptop to phone: https://imgur.com/a/7iHwv
> >>>>> 18GB file from laptop to ssd: https://imgur.com/a/40Q6V
> >>>>> multiple small files from laptop to phone: https://imgur.com/a/B4v4y
> >>>>> multiple small files from laptop to ssd: https://imgur.com/a/mDiMu
> >>>>>
> >>>>> The files are missing timestamps but the originals were taken with
> scrot and have timestamps available here: https://nofile.io/f/
> mzKnkpM9CyC/stats.tar.gz2
> >>>>>
> >>>>> as far as why there's such high deletions? I can't say I'm only
> using cp.
> >>>>
> >>>> I assume that md99 is for a file-based swap-space, such as
> >>>> via /var/swap0 file. (As a side note I warn about bugzilla
> >>>> 206048 for such contexts.) Otherwise please describe how
> >>>> md99 is created. (Below I assume the swap-space usage of
> >>>> md99.)
> >>>>
> >>>> The only other device that your pictures show is your
> >>>> NVMe device nvd0.
> >>>>
> >>>> No picture shows a device for the LG v30 when it is mentioned
> >>>> above as being copied to or from. How is it that there is no
> >>>> mounted device shown for the LG v30?
> >>>>
> >>>> No picture shows a device for the SSD when it is mentioned
> >>>> above as being copied to or from. How is it that there
> >>>> is no mounted device shown for the SSD?
> >>>>
> >>>> Without a device displayed for the LG-v30/SSD there is nothing
> >>>> displayed for its reads or writes. This makes the gstat -pd
> >>>> useless.
> >>>>
> >>>> May be the p needs to be omitted for some reason? gstat -d
> >>>>
> >>>> ===
> >>>> Mark Millard
> >>>> markmi at dsl-only.net
> >>>
> >>> You are correct that md99 is a file backed swap disk, I am aware of
> the issues but I had to test some things out.
> >>>
> >>> Those tests earlier was a huge time sink.
> >>> Here's the dmesg output from earlier:
> >>> . . .
> >>> ----------------------------------
> >>>
> >>> I don't know why gstat isn't showing the info u are looking for.
> >>>
> >>> You said earlier that something was getting deleted,
> >>> I don't know what could be causing that.
> >>
> >> Those "deletes" are more commonly called TRIM on SSD's.
> >> FreeBSD called them, BIO_DELETE as I remember.
> >>
> >> Those deletes have nothing to do with umounting, deleteing
> >> devices, etc.
> >>
> >>> I've provided tons of debug out, logs, pciconf, dmesg, etc...
> >>> Even say forget the mobile device and go from
> >>> zpool
> >>
> >> From which I've not been able to figure out the kind of
> >> information that I'm looking for.
> >>
> >> Just because a device is present, does not mean that it
> >> is available as a file system. I'm more interested in
> >> how the file systems are made available --or if some
> >> non-file system way of working is involved.
> >>
> >>> Are you saying that there's something misconfigured on my end?
> >>> What other info do you need me to provide to try to figure out
> >>> what's going on or why I'm getting these transfer rates?
> >>
> >> I'm saying I still can not tell how you make the SSD or the
> >> LGv 30 available to FreeBSD (mount?). Or why no matching
> >> mounted device shows up in gstat's display.
> >>
> >> If you wish to keep trying to help me help you, . . .
> >>
> >> Please show how you make the file system on the
> >> SSD available to FreeBSD: what FreeBSD commands make
> >> the device available for use. (I'd guess that mount
> >> is used or that something like /etc/fstab is used
> >> to do mounts more implicitly.)
> >>
> >> Please show how you make the LG v30 available to FreeBSD:
> >> what FreeBSD commands make the device available for
> >> use. (I'd guess that mount is used or that something like
> >> /etc/fstab is used to do mounts more implicitly.)
> >>
> >> (I would expect these are mount commands, or at least
> >> involve mount commands/calls. Some of the following makes
> >> that presumption.)
> >>
> >> For each of those: with the device available show the
> >> output of:
> >>
> >> mount
> >>
> >> and of:
> >>
> >> df -m
> >>
> >> Similarly, show the exact commands used to make the
> >> copies to and from the SSD. Show the exact commands
> >> used to make the copies to and from the LG v30.
> >>
> >> (You can for now stop the commands early or just
> >> not start any that would take a long time.)
> >>
> >> I'm looking for a way to get information similar to
> >> what I expected gstat to show. I'd expect a mounted
> >> file system but may be something else is involved?
> >
> [Extracted from a reply to a Jon Brawn post.]
> > _at_Mark Millard
> > I use sysutils/simple-mtpfs to mount the android device.
> > when I mount the phone through USB this is the relevant section:
> > /dev/fuse                                   356311 78912 277398    22%
> > /mnt
> >
> > That's the most complicated mount process that I use,
> > for the ssd it's just a simple mount /dev/device /mnt
> > relevant output:
> > /dev/da0                                    923913 121450 728550    14%
> > /mnt
> >
> > Can you tell me what information you're looking for so that I can gather
> it
> > all up and send it.
>
>
> I do not find a /usr/ports/sysutils/simple-mtpfs . But I
> do find:
>
> # ls -lTd /usr/ports/sysutils/*simple*
> drwxr-xr-x  3 root  wheel  512 Sep 23 19:51:59 2017
> /usr/ports/sysutils/fusefs-simple-mtpfs
>
> And that was very important to know.
>
> I had to look it up but this means that you are using
> the Media Transfer Protocol (MTP) to transfer files from
> the LG v30. It also means that part of that involves use
> of the Filesystem in USErspace (FUSE). The two would be
> likely to add overheads that slow things down (extra
> stages/steps in getting the bytes copied).
>
> [Apparently MTP is an extension to the Picture Transfer
> Protocol (PTP) communications protocol. MTP has been
> standardized as a full-fledged USB device class but
> originated with Microsoft.]
>
> [MTP use involves putting the Android device into MTP mode,
> frequently referenced in the Android UI via a selection
> of "USB for file transfer" someplace in the user
> interface (according to what I read).]
>
> One example of why FreeBSD and sysutils/fusefs-simple-mtpfs
> and sysutils/fusefs-libs and multimedia/libmtp might be
> slower than some linux software is:
>
> It appears that some Linux software has implemented "USB
> 'Zerocopy' support found in recent Linux kernel" (and,
> so, avoid user-space vs. kernel space data copying and
> some context switching).
>
> I doubt that FreeBSD and its sysutils/fusefs-simple-mtpfs
> and sysutils/fusefs-libs and multimedia/libmtp combination
> happen to do that. This difference likely would make a
> notable speed difference for the transfer and save to local
> file system (if my doubt is correct).
>
> It is not surprising that the speeds would be far less
> than the experiments that I reported.
>
>
> I do not have a context to experiment with the
> consequences of using sysutils/fusefs-simple-mtpfs
> and what it requires (fusefs-libs and libmtp). I'm
> afraid I'll not be able to be much help relative to
> the LG v30 MTP-based transfer performance
> investigations.
>
>
> That leaves the SSD context. Is there anything that
> you still want to investigate relative to the SSD
> usage? Or is there no point because of the LG v30
> status?
>
> ===
> Mark Millard
> markmi at dsl-only.net
>
>
> OK, forget the android specific usb issues for now;
there is quite a few layers of crud between the laptop and the android
device.

I have this SSD.

What are some tests that you'd like to run to test the speeds
on my end?