Re: cp(1) of large files is causing 100% CPU utilization and poor transfer

Just fyi, I've reproduced the problem.
All I did was create a 20Gbyte file
on UFS on a slow (4Gbyte or RAM,
slow spinning disk) laptop.
(The UFS file system is just what the installer creates these days.)

cp still hasn't finished and is definitely
taking a looott longer than dd did.

I'll start drilling down later to-day.

I'll admit doing lots of testing of copy_file_range(2)
with large sparse files, but I may have missed testing
a large non-sparse file.

rick
ps: I've added Kostik and Kirk to the cc.


________________________________________
From: owner-freebsd-current_at_freebsd.org <owner-freebsd-current_at_freebsd.org> on behalf of Alan Somers <asomers_at_freebsd.org>
Sent: Saturday, January 2, 2021 11:30 AM
To: Matthias Apitz; FreeBSD CURRENT
Subject: Re: cp(1) of large files is causing 100% CPU utilization and poor transfer

CAUTION: This email originated from outside of the University of Guelph. Do not click links or open attachments unless you recognize the sender and know the content is safe. If in doubt, forward suspicious emails to IThelp_at_uoguelph.ca


On Sat, Jan 2, 2021 at 9:12 AM Matthias Apitz <guru_at_unixarea.de> wrote:

> El día sábado, enero 02, 2021 a las 09:06:24a. m. -0700, Alan Somers
> escribió:
>
> > > As I said, it can be reproduced using only the local file system. This
> > > was setup recently on a SSD:
> > >
> > > # dmesg | grep ada0
> > > ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
> > > ada0: <TS512GMTS430S R0906A> ACS-2 ATA SATA 3.x device
> > > ada0: Serial Number F995890846
> > > ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 1024bytes)
> > > ada0: Command Queueing enabled
> > > ada0: 488386MB (1000215216 512 byte sectors)
> > >
> > > and by this procedure:
> > >
> > > # gpart create -s gpt ada0
> > > # gpart add -t freebsd-boot -s 512k -a4k -l ssdboot ada0
> > > # gpart bootcode -b /boot/pmbr -p /boot/gptboot -i1 ada0
> > > # gpart add -t freebsd-ufs -l ssdrootfs -b 1m -s 2g ada0
> > > # gpart add -t freebsd-ufs -l ssdvarfs -a 1m -s 2g ada0
> > > # gpart add -t freebsd-ufs -l ssdusrfs -a 1m ada0
> > > # newfs -U -t /dev/gpt/ssdrootfs
> > > # newfs -U -t /dev/gpt/ssdvarfs
> > > # newfs -U -t /dev/gpt/ssdusrfs
> > >
> > > # gpart show -l ada0
> > > =>        40  1000215136  ada0  GPT  (477G)
> > >           40        1024     1  ssdboot  (512K)
> > >         1064         984        - free -  (492K)
> > >         2048     4194304     2  ssdrootfs  (2.0G)
> > >      4196352     4194304     3  ssdvarfs  (2.0G)
> > >      8390656    16777216     4  ssdswap  (8.0G)
> > >     25167872   975046656     5  ssdusrfs  (465G)
> > >   1000214528         648        - free -  (324K)
> > >
> > > # mount -t ufs
> > > /dev/gpt/ssdrootfs on / (ufs, local, soft-updates)
> > > /dev/gpt/ssdvarfs on /var (ufs, local, soft-updates)
> > > /dev/gpt/ssdusrfs on /usr (ufs, local, soft-updates)
> > >
> > > When I run in the /usr fs the command
> > >
> > > # cp -p guru-20210102.tar.gz xxx
> > >
> > > it copies around 168M per minute.
> > >
>
> > Is that copying from /usr to /usr, or from /usr to /var or /?
>
> # cd /home/backups
> # cp -p guru-20210102.tar.gz xxx
>
> i.e. from /usr to /usr.
>
>         matthias
>

Ok, let's narrow this down.  Could you please run the command with the
attached D script ?
sudo dtrace -s copy_file_range.d -c "cp -p guru-20210102.tar.gz xxx"
_______________________________________________
freebsd-current_at_freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"