Steven Hartland wrote: SH> I'm not clear what you rolled back there as r255173 has ntothing to do SH> with this. Could you clarify r255173 with you patch from email dated Tue, 17 Sep 2013 23:53:12 +0100 with subject Re: ZFS secondarycache on SSD problem on r255173 Errors wich we gets is in arcstats count not in messages, and was desribed some time ago in mails be me and Dmitriy Makarov with subject's ZFS L2ARC - incorrect size and abnormal system load on r255173 On r255173 without patch and with vfs.zfs.max_auto_ashift=9 when added to pool 2 ssd as caches get cache gpt/cache1 ONLINE 0 0 0 block size: 512B configured, 4096B native gpt/cache2 ONLINE 0 0 0 block size: 512B configured, 4096B native Same message we seen with default vfs.zfs.max_auto_ashift Will wait some time to see how it works. SH> Any errors recorded in /var/log/messages? SH> SH> Could you add code to record the none zero value of zio->io_error in SH> l2arc_read_done as this may give some indication of the underlying SH> issue. SH> SH> Additionally could always put a panic in that code path too and then SH> create a dump so the details can be fully exhamined. SH> SH> In terms of the slowness thats going to be a side effect of the cache SH> failures. SH> SH> Oh could you also confirm that the issue doesn't exist if you SH> 1. Exclude r255753 SH> 2. Set vfs.zfs.max_auto_ashift=9 SH> SH> Regards SH> Steve SH> ----- Original Message ----- SH> From: "Vitalij Satanivskij" <satan_at_ukr.net> SH> To: "Steven Hartland" <killing_at_multiplay.co.uk> SH> Cc: "Vitalij Satanivskij" <satan_at_ukr.net>; "Dmitriy Makarov" <supportme_at_ukr.net>; "Justin T. Gibbs" <gibbs_at_freebsd.org>; "Borja SH> Marcos" <borjam_at_sarenet.es>; <freebsd-current_at_freebsd.org> SH> Sent: Wednesday, October 16, 2013 3:10 PM SH> Subject: Re: ZFS secondarycache on SSD problem on r255173 SH> SH> SH> > Yes SH> > SH> > We have 15 servers, all of them have problem while using with patch fo ashift, sh we rollback path (for r255173) SH> > and all of them works for a week without that's problem's. Yesterday one of of servers was updated to stable/10 (beta1) SH> > SH> > wich include patch and after around 12 hours of works l2arc begin et errors like that SH> > SH> > kstat.zfs.misc.arcstats.l2_cksum_bad SH> > kstat.zfs.misc.arcstats.l2_io_error SH> > SH> > SH> > For now patch disabled in ower production. SH> > SH> > SH> > Please note we have very heavy load on zfs pool so 90GB arc and 3x180Gb L2arc have very big hit's on it on it. SH> > SH> > SH> > SSD used for cache's is intel ssd 530 series smart for all devices in in normal states's SH> > no bad values on it. SH> > SH> > Steven Hartland wrote: SH> > SH> Have you confirmed the ashift changes are the actual cause of this SH> > SH> by backing out just those changes and retesting on the same hardware. SH> > SH> SH> > SH> Also worth checking your disks smart values to confirm there are no SH> > SH> visible signs of HW errors. SH> > SH> SH> > SH> Regards SH> > SH> Steve SH> > SH> SH> > SH> ----- Original Message ----- SH> > SH> From: "Vitalij Satanivskij" <satan_at_ukr.net> SH> > SH> To: "Dmitriy Makarov" <supportme_at_ukr.net> SH> > SH> Cc: "Steven Hartland" <killing_at_multiplay.co.uk>; "Justin T. Gibbs" <gibbs_at_freebsd.org>; "Borja Marcos" <borjam_at_sarenet.es>; SH> > SH> <freebsd-current_at_freebsd.org> SH> > SH> Sent: Wednesday, October 16, 2013 9:01 AM SH> > SH> Subject: Re: ZFS secondarycache on SSD problem on r255173 SH> > SH> SH> > SH> SH> > SH> > Hello. SH> > SH> > SH> > SH> > Patch brocke cache functionality. SH> > SH> > SH> > SH> > Look at's Dmitriy's mail from Mon, 07 Oct 2013 21:09:06 +0300 SH> > SH> > SH> > SH> > With subject ZFS L2ARC - incorrect size and abnormal system load on r255173 SH> > SH> > SH> > SH> > As patch alredy in head and BETA it's not good. SH> > SH> > SH> > SH> > Yesterday we update one machine up to beta1 and forgot about patch. So 12 Hours and cache broken... :(( SH> > SH> > SH> > SH> > SH> > SH> > SH> > SH> > Dmitriy Makarov wrote: SH> > SH> > DM> The attached patch by Steven Hartland fixes issue for me too. Thank you! SH> > SH> > DM> SH> > SH> > DM> SH> > SH> > DM> --- Исходное сообщение --- SH> > SH> > DM> От кого: "Steven Hartland" < killing_at_multiplay.co.uk > SH> > SH> > DM> Дата: 18 сентября 2013, 01:53:10 SH> > SH> > DM> SH> > SH> > DM> ----- Original Message ----- SH> > SH> > DM> From: "Justin T. Gibbs" < SH> > SH> > DM> SH> > SH> > DM> --- SH> > SH> > DM> Дмитрий Макаров SH> > SH> > DM> _______________________________________________ SH> > SH> > DM> freebsd-current_at_freebsd.org mailing list SH> > SH> > DM> http://lists.freebsd.org/mailman/listinfo/freebsd-current SH> > SH> > DM> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org" SH> > SH> > SH> > SH> SH> > SH> SH> > SH> ================================================ SH> > SH> This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the SH> > event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any SH> > information contained in it. SH> > SH> SH> > SH> In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 SH> > SH> or return the E.mail to postmaster_at_multiplay.co.uk. SH> > SH> SH> > SH> _______________________________________________ SH> > SH> freebsd-current_at_freebsd.org mailing list SH> > SH> http://lists.freebsd.org/mailman/listinfo/freebsd-current SH> > SH> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org" SH> > SH> SH> SH> ================================================ SH> This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. SH> SH> In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 SH> or return the E.mail to postmaster_at_multiplay.co.uk. SH> SH> _______________________________________________ SH> freebsd-current_at_freebsd.org mailing list SH> http://lists.freebsd.org/mailman/listinfo/freebsd-current SH> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"Received on Wed Oct 16 2013 - 13:57:19 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:43 UTC