Peter Losher wrote: > I have been poking around with our 16-port SATA RAID chassis (in JBOD > mode) with a fresh RELENG_7 internal build from Saturday. I only had > half the disks {da0-7) at first, when I brought over the second set > (da8-15), I destroyed the existing pool and then tried to create a new > one with all 16 disks, and encountered this error: > > -=- > # zpool create tank raidz da{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15} > cannot create 'tank': one or more vdevs refer to the same device > -=- > > I can create one with the second set of disks (and destroy/create again) > but somehow I have some old vdevs sitting around which is preventing me > from using the first set of disks. How can I clear them out? O.k. poked at this some more this evening and I see what the problem might be. The SATA RAID chassis (in JBOD mode) is presenting the 16 drives as two sets of 8 drives each (it has two FC connections, one for each set - incidentally in RAID mode, it just needs one to cover all 16 disks as it's able to bridge the two controllers together). Those two FC connections are plugged into a dual port LSI Logic FC HBA: -=- mpt0: <Dual LSILogic FC7X04X 4Gb/s FC PCI-X Adapter> port 0x2400-0x24ff mem 0xb8a20000-0xb8a23fff,0xb8a00000-0xb8a0ffff irq 24 at device 3.0 on pci8 mpt0: [ITHREAD] mpt0: MPI Version=1.5.10.0 mpt1: <Dual LSILogic FC7X04X 4Gb/s FC PCI-X Adapter> port 0x2000-0x20ff mem 0xb8a24000-0xb8a27fff,0xb8a10000-0xb8a1ffff irq 27 at device 3.1 on pci8 mpt1: [ITHREAD] mpt1: MPI Version=1.5.10.0 -=- so this disks show up as such: -=- da0 at mpt0 bus 0 target 0 lun 0 da0: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da0: 100.000MB/s transfers da0: Command Queueing Enabled da0: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da1 at mpt0 bus 0 target 0 lun 1 da1: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da1: 100.000MB/s transfers da1: Command Queueing Enabled da1: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da2 at mpt0 bus 0 target 0 lun 2 da2: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da2: 100.000MB/s transfers da2: Command Queueing Enabled da2: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da3 at mpt0 bus 0 target 0 lun 3 da3: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da3: 100.000MB/s transfers da3: Command Queueing Enabled da3: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da4 at mpt0 bus 0 target 0 lun 4 da4: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da4: 100.000MB/s transfers da4: Command Queueing Enabled da4: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da5 at mpt0 bus 0 target 0 lun 5 da5: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da5: 100.000MB/s transfers da5: Command Queueing Enabled da5: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da6 at mpt0 bus 0 target 0 lun 6 da6: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da6: 100.000MB/s transfers da6: Command Queueing Enabled da6: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da7 at mpt0 bus 0 target 0 lun 7 da7: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da7: 100.000MB/s transfers da7: Command Queueing Enabled da7: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da8 at mpt1 bus 0 target 0 lun 0 da8: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da8: 100.000MB/s transfers da8: Command Queueing Enabled da8: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da9 at mpt1 bus 0 target 0 lun 1 da9: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da9: 100.000MB/s transfers da9: Command Queueing Enabled da9: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da10 at mpt1 bus 0 target 0 lun 2 da10: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da10: 100.000MB/s transfers da10: Command Queueing Enabled da10: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da11 at mpt1 bus 0 target 0 lun 3 da11: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da11: 100.000MB/s transfers da11: Command Queueing Enabled da11: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da12 at mpt1 bus 0 target 0 lun 4 da12: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da12: 100.000MB/s transfers da12: Command Queueing Enabled da12: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da13 at mpt1 bus 0 target 0 lun 5 da13: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da13: 100.000MB/s transfers da13: Command Queueing Enabled da13: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da14 at mpt1 bus 0 target 0 lun 6 da14: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da14: 100.000MB/s transfers da14: Command Queueing Enabled da14: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) da15 at mpt1 bus 0 target 0 lun 7 da15: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device da15: 100.000MB/s transfers da15: Command Queueing Enabled da15: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) -=- Two identical sets of 8 disks, the only difference is that one set is presented over mpt0, the other over mpt1. I suspect ZFS is treating this as just one set of eight vdevs as I can create a pool with either the first set of eight or the second set of eight, just not with BOTH sets of disks. Hope this helps explain things a bit better. Best Wishes - Peter -- Peter_Losher_at_isc.org | ISC | OpenPGP 0xE8048D08 | "The bits must flow"
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:20 UTC