CAM Shingled Disk support patches available

From: Kenneth D. Merry <ken_at_FreeBSD.ORG>
Date: Wed, 18 Nov 2015 12:13:09 -0500
I have work in progress patches to add SMR (Shingled Magnetic Recording)
support to CAM and GEOM here:

FreeBSD/head as of SVN revision 290997:

https://people.freebsd.org/~ken/cam_smr.head.20151117.1.txt

FreeBSD stable/10 as of SVN revision 290995:

https://people.freebsd.org/~ken/cam_smr.stable10.20151117.1.txt

This includes support for Host Managed, Host Aware and Drive Managed SMR
drives that are either SCSI (ZBC) or ATA (ZAC) attached via a SAS
controller.  This does not include support for SMR ATA drives attched via
an ATA controller.  Also, I have not yet figured out how to properly detect
a Host Managed ATA drive, so this code won't do that.

The big drive vendors are moving to SMR for at least some of their drives.
The primary challenge with SMR is that it requires writing a relatively
large zone sequentially starting at the beginning of the zone.  The usual
zone size is 256MB.  It is conceptually almost like having a 256MB sector
size.

We (Spectra Logic) are working on ZFS changes that will use this CAM and
GEOM infrastructure to make ZFS play well with SMR drives.  Those changes
aren't yet done.

The patches linked above include:
 o A new 'camcontrol zone' command that allows displaying and managing
   drive zones via SCSI/ATA passthrough.
 o A new zonectl(8) utility that uses the new DIOCZONECMD ioctl to display
   and manage zones via the da(4) (and later ada(4)) driver.
 o Changes to diskinfo -v to display the zone mode of a drive.
 o A new disk zone API, sys/sys/disk_zone.h.
 o A new bio type, BIO_ZONE, and modifications to GEOM to support it.  This
   new bio will allow filesystems to query zone support in a drive and
   manage zoned drives.
 o Extensive modifications to the da(4) driver to handle probing SCSI and
   SATA behind SAS SMR drives.
 o Additional CAM CDB building functions for zone commands.

The current issues that need to be addressed are:
 o The da(4) driver now has 6 additional probe states, 5 of which are
   needed for probing ATA drives behind SAS controllers.  I have not yet
   added support for BIO_ZONE bios to ada(4), but it will be very similar
   to the da(4) driver version.  The ATA probe code needs to be pulled
   out of the da(4) driver and changed into a form that will allow it to
   work for either the ada(4) or da(4) driver.  Otherwise we'll have a fair
   amount of code duplication between the two drivers.

 o There is a reasonable amount of code duplication between 'camcontrol zone'
   and zonectl(8).  This was done for speed / expediency's sake, but it may
   be possible to make more of the code common there.

 o In order to add the new BIO_ZONE bio command, I had to change the bio_cmd
   field in struct bio from a uint8_t to a uint16_t.  This will cause
   binary compatibility problems with any 3rd party loadable modules.
   Advice on how to handle this would be welcome.

 o In the process of developing these changes, I discovered that the
   dxfer_len paramter for scsi_ata_pass_16() was too small (uint16_t, and
   it needed to be uint32_t).  I increased it, but that will potentially
   cause a binary incompatibility problem with any existing applications
   that use the current API via libcam.  Advice on how to handle that
   would be welcome.

If you look through the code, you'll notice that the disk_zone.h API is
separate from the SCSI and ATA APIs.  The intent is to allow filesystems
and other consumers of the API to just talk to the disk zone API without
dealing with the SCSI and ATA specifics.  Another reason behind all of this
is that even though the SCSI ZBC and ATA ZAC specs were developed in
concert, and are intended to be functionally identical, they are still SCSI
and ATA.  As usual, SCSI is big endian and ATA is little endian.  So to
present a common API to the filesystem, we give all of the zone data back
in native byte order, regardless of the underlying device protocol.

Another thing to note is the extensive use of ATA passthrough in the da(4)
driver.  This is necessary because the SCSI SAT (SCSI to ATA Translation)
specification has not yet caught up with translating SCSI zone commands
(ZBC) to ATA zone commands (ZAC).  So, until the spec is updated and LSI
and other vendors update their SCSI to ATA translation layers, we'll have
to use the ATA version of the commands when talking to ATA drives via SAS
controllers.

I have only tested the code so far with Seagate SATA Drive Managed and Host
Aware drives.  I would appreciate testing with any drives.  (And testing to
make sure that the patches don't cause problems with existing hardware.)
Right now, all you can really do is manage the zones manually using
camcontrol(8) or zonectl(8).  Automatic management will come with the ZFS
changes.  (Or changes to other filesysems if people want to do it.)

If you have a SATA Host Aware drive, in theory camcontrol(8) should allow
you to manage the drive if you have it attached to a SATA controller.

Here is an example of some of the commands.

Get general zoning information:

[root_at_sm4u-1 ~]# zonectl -c params -d /dev/da21
Zone Mode: Host Aware
Command support: Report Zones, Open, Close, Finish, Reset Write Pointer
Unrestricted Read in Sequential Write Required Zone (URSWRZ): No
Optimal Number of Open Sequential Write Preferred Zones: 128
Optimal Number of Non-Sequentially Written Sequential Write Preferred Zones: 8
Maximum Number of Open Sequential Write Required Zones: Unlimited

Look at information from the da(4) driver:

[root_at_sm4u-1 ~]# sysctl kern.cam.da.21
kern.cam.da.21.delete_method: NONE
kern.cam.da.21.delete_max: 1081344
kern.cam.da.21.minimum_cmd_size: 6
kern.cam.da.21.sort_io_queue: -1
kern.cam.da.21.zone_mode: Host Aware
kern.cam.da.21.zone_support: Report Zones, Open, Close, Finish, Reset Write Pointer
kern.cam.da.21.optimal_seq_zones: 128
kern.cam.da.21.optimal_nonseq_zones: 8
kern.cam.da.21.max_seq_zones: 4294967295
kern.cam.da.21.error_inject: 0

Display all of the zones with zonectl(8):

[root_at_sm4u-1 ~]# zonectl -d /dev/da21 -c rz
29809 zones, Maximum LBA 0x3a3812aaf (15628053167)
Zone lengths and types may vary
  Start LBA  Length       WP LBA      Zone Type      Condition      Sequential             Reset
          0, 524288,     0x80000,  Conventional,           NWP,     Sequential,  No Reset Needed
    0x80000, 524288,    0x100000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x100000, 524288,    0x180000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x180000, 524288,    0x200000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x200000, 524288,    0x280000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x280000, 524288,    0x300000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x300000, 524288,    0x380000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x380000, 524288,    0x400000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x400000, 524288,    0x480000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x480000, 524288,    0x500000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x500000, 524288,    0x580000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x580000, 524288,    0x600000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x600000, 524288,    0x680000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x680000, 524288,    0x700000,  Conventional,           NWP,     Sequential,  No Reset Needed
   0x700000, 524288,    0x780000,  Conventional,           NWP,     Sequential,  No Reset Needed
[ ... ]
 0x1f00000, 524288,   0x1f80000,  Conventional,           NWP,     Sequential,  No Reset Needed
  0x1f80000, 524288,   0x2000000,  Conventional,           NWP,     Sequential,  No Reset Needed
  0x2000000, 524288,   0x2080000, Seq Preferred,          Full,     Sequential,  No Reset Needed
  0x2080000, 524288,   0x2100000, Seq Preferred,          Full,     Sequential,  No Reset Needed
  0x2100000, 524288,   0x2180000, Seq Preferred,          Full,     Sequential,  No Reset Needed
  0x2180000, 524288,   0x2200000, Seq Preferred,          Full,     Sequential,  No Reset Needed
  0x2200000, 524288,   0x2280000, Seq Preferred,          Full,     Sequential,  No Reset Needed
  0x2280000, 524288,   0x2300000, Seq Preferred,          Full,     Sequential,  No Reset Needed
[ ... ]

Use camcontrol zone to reset the write pointer for the first shingled zone
listed above:

[root_at_sm4u-1 ~]# camcontrol zone da21 -v -c rwp -l 0x2000000

Use camcontrol zone to ask the drive to report empty zones:

[root_at_sm4u-1 ~]# camcontrol zone da21 -v -c rz -o empty
1 zones, Maximum LBA 0x3a3812aaf (15628053167)
Zone lengths and types may vary
  Start LBA  Length       WP LBA      Zone Type      Condition      Sequential             Reset
  0x2000000, 524288,   0x2000000, Seq Preferred,         Empty,     Sequential,  No Reset Needed

Get information on a Host Aware drive:

root_at_sm4u-1 ~]# diskinfo -v da21
da21
        512             # sectorsize
        8001563222016   # mediasize in bytes (7.3T)
        15628053168     # mediasize in sectors
        4096            # stripesize
        0               # stripeoffset
        972801          # Cylinders according to firmware.
        255             # Heads according to firmware.
        63              # Sectors according to firmware.
                    Z84008NY    # Disk ident.
        enc_at_5003048001f311fd/elmtype_at_array_device/slot_at_22       # Physical path
        Host Aware      # Zone Mode

Get information on a drive managed drive:

[root_at_sm4u-1 ~]# diskinfo -v da20
da20
        512             # sectorsize
        8001563222016   # mediasize in bytes (7.3T)
        15628053168     # mediasize in sectors
        4096            # stripesize
        0               # stripeoffset
        972801          # Cylinders according to firmware.
        255             # Heads according to firmware.
        63              # Sectors according to firmware.
                    Z8405938    # Disk ident.
        enc_at_5003048001f311fd/elmtype_at_array_device/slot_at_21       # Physical path
        Drive Managed   # Zone Mode

Get information on a non-zoned drive:

[root_at_sm4u-1 ~]# diskinfo -v da4
da4
        512             # sectorsize
        100030242816    # mediasize in bytes (93G)
        195371568       # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        12161           # Cylinders according to firmware.
        255             # Heads according to firmware.
        63              # Sectors according to firmware.
        124903574F36            # Disk ident.
        enc_at_5003048001f311fd/elmtype_at_array_device/slot_at_5        # Physical path
        Not Zoned       # Zone Mode

Testing and comments are welcome.

Thanks,

Ken
-- 
Kenneth Merry
ken_at_FreeBSD.ORG
Received on Wed Nov 18 2015 - 16:13:12 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:01 UTC