Re: ZFS - Abyssal slow on copying

Am Sun, 2 Oct 2016 15:30:41 -0400
Allan Jude <allanjude_at_freebsd.org> schrieb:

> On 2016-10-02 15:25, O. Hartmann wrote:
> > 
> > Running 12-CURRENT (FreeBSD 12.0-CURRENT #32 r306579: Sun Oct  2 09:34:50 CEST 2016
> > ), I have a NanoBSD setup which creates an image for a router device.
> > 
> > The problem I face is related to ZFS. The system has a system's SSD (Samsung 850 Pro,
> > 256GB) which has an UFS filesystem. Aditionally, I have also a backup and a data HDD,
> > both WD, one 3 TB WD RED Pro, on 4 TB WD RED (the backup device). Both the sources for
> > the NanoBSD and the object tree as well as the NANO_WORLDDIR are residing on the 3 TB
> > data drive. 
> > 
> > The box itself has 8 GB RAM. When it comes to create the memory disk, which is ~ 1,3
> > GB in size, the NanoBSD script starts creating the memory disk and then installing
> > world into this memory disk. And this part is a kind of abyssal in terms of the speed.
> > 
> > The drive sounds like hell, the heads are moving rapidly. The copy speed is incredibly
> > slow compared to another box I usually use in the lab with UFS filesystem only
> > (different type of HDD).
> > 
> > The whole stuff the nanbsd is installed from and to is on a separate ZFS partition,
> > but in the same pool as everything else. When I first setup the new partitions, I
> > switched on deduplication, but I quickly deactivated it, because it had a tremendous
> > impact on the working speed and memory consumption on that box. But something seems
> > not right since then - as I initially described, the copy/initialisation
> > speed/bandwith is abyssal. Well, I also fear that I did something wrong when I firt
> > initialised the HDD - there is this 125bytes/4k block discussion and I do not know
> > how to check whether I'm affected to that or not (or even causing the problems) and
> > how to check whether DEDUPLICATION is definitely OFF (apart from the usual stuff list
> > features via "zfs get all").
> > 
> > As an example: the nanbosd script takes ~ 1 minute to copy /boot/loader from source to
> > memory disk and the HDD makes sounds like hell and close to loosing the r/w heads. On
> > other boxes this task is done in a blink of an eye ...
> > 
> > Thanks for your patience,
> > 
> > Regards,
> > oh
> >   
> 

Hello Jude.

Thank you for your answer.

> Turning deduplication off, only stops new blocks from being
> deduplicated. Any data written while deduplication was on, are still
> deduplicated. You would need to zfs send | zfs recv, or
> backup/destroy/restore to get the data back to normal.

All right, that confirms my fear - I had dedup on for a while when experimenting ... 

> 
> If the drive is making that much noise, have you considered that the
> drive might be failing?

The drive seems all right, I think it's the reading/writing of many small blocks at once
when copying from that specific ZFS partition containing the nanbsd installation with
dedup on once. The noise is "classical", but unusual in that specific task when supposed
not to do so many read/writes. But, anyway, it is an optimistic guess and somehow also
wishful thinking ;-) I will take a drive failure also in consideration - I hopefully have
backups.

Kind reagrds,

Oliver
>