On Mon, Oct 29, 2007 at 12:18:50AM -0700, Peter Losher wrote: > Peter Losher wrote: > > > I have been poking around with our 16-port SATA RAID chassis (in JBOD > > mode) with a fresh RELENG_7 internal build from Saturday. I only had > > half the disks {da0-7) at first, when I brought over the second set > > (da8-15), I destroyed the existing pool and then tried to create a new > > one with all 16 disks, and encountered this error: > > > > -=- > > # zpool create tank raidz da{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15} > > cannot create 'tank': one or more vdevs refer to the same device > > -=- > > > > I can create one with the second set of disks (and destroy/create again) > > but somehow I have some old vdevs sitting around which is preventing me > > from using the first set of disks. How can I clear them out? > > O.k. poked at this some more this evening and I see what the problem > might be. The SATA RAID chassis (in JBOD mode) is presenting the 16 > drives as two sets of 8 drives each (it has two FC connections, one for > each set - incidentally in RAID mode, it just needs one to cover all 16 > disks as it's able to bridge the two controllers together). Those two > FC connections are plugged into a dual port LSI Logic FC HBA: > > -=- > mpt0: <Dual LSILogic FC7X04X 4Gb/s FC PCI-X Adapter> port 0x2400-0x24ff > mem 0xb8a20000-0xb8a23fff,0xb8a00000-0xb8a0ffff irq 24 at device 3.0 on pci8 > mpt0: [ITHREAD] > mpt0: MPI Version=1.5.10.0 > mpt1: <Dual LSILogic FC7X04X 4Gb/s FC PCI-X Adapter> port 0x2000-0x20ff > mem 0xb8a24000-0xb8a27fff,0xb8a10000-0xb8a1ffff irq 27 at device 3.1 on pci8 > mpt1: [ITHREAD] > mpt1: MPI Version=1.5.10.0 > -=- > > so this disks show up as such: > > -=- > da0 at mpt0 bus 0 target 0 lun 0 > da0: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da0: 100.000MB/s transfers > da0: Command Queueing Enabled > da0: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da1 at mpt0 bus 0 target 0 lun 1 > da1: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da1: 100.000MB/s transfers > da1: Command Queueing Enabled > da1: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da2 at mpt0 bus 0 target 0 lun 2 > da2: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da2: 100.000MB/s transfers > da2: Command Queueing Enabled > da2: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da3 at mpt0 bus 0 target 0 lun 3 > da3: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da3: 100.000MB/s transfers > da3: Command Queueing Enabled > da3: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da4 at mpt0 bus 0 target 0 lun 4 > da4: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da4: 100.000MB/s transfers > da4: Command Queueing Enabled > da4: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da5 at mpt0 bus 0 target 0 lun 5 > da5: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da5: 100.000MB/s transfers > da5: Command Queueing Enabled > da5: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da6 at mpt0 bus 0 target 0 lun 6 > da6: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da6: 100.000MB/s transfers > da6: Command Queueing Enabled > da6: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da7 at mpt0 bus 0 target 0 lun 7 > da7: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da7: 100.000MB/s transfers > da7: Command Queueing Enabled > da7: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da8 at mpt1 bus 0 target 0 lun 0 > da8: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da8: 100.000MB/s transfers > da8: Command Queueing Enabled > da8: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da9 at mpt1 bus 0 target 0 lun 1 > da9: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da9: 100.000MB/s transfers > da9: Command Queueing Enabled > da9: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da10 at mpt1 bus 0 target 0 lun 2 > da10: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da10: 100.000MB/s transfers > da10: Command Queueing Enabled > da10: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da11 at mpt1 bus 0 target 0 lun 3 > da11: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da11: 100.000MB/s transfers > da11: Command Queueing Enabled > da11: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da12 at mpt1 bus 0 target 0 lun 4 > da12: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da12: 100.000MB/s transfers > da12: Command Queueing Enabled > da12: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da13 at mpt1 bus 0 target 0 lun 5 > da13: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da13: 100.000MB/s transfers > da13: Command Queueing Enabled > da13: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da14 at mpt1 bus 0 target 0 lun 6 > da14: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da14: 100.000MB/s transfers > da14: Command Queueing Enabled > da14: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > da15 at mpt1 bus 0 target 0 lun 7 > da15: <Seagate ST3400633NS R001> Fixed Direct Access SCSI-3 device > da15: 100.000MB/s transfers > da15: Command Queueing Enabled > da15: 381554MB (781422768 512 byte sectors: 255H 63S/T 48641C) > -=- > > Two identical sets of 8 disks, the only difference is that one set is > presented over mpt0, the other over mpt1. I suspect ZFS is treating > this as just one set of eight vdevs as I can create a pool with either > the first set of eight or the second set of eight, just not with BOTH > sets of disks. > > Hope this helps explain things a bit better. Let me get this right. Those are 16 physically different disks? Not 8 disks visible through two FC path? Can you show: # apply "camcontrol inquiry da%1 -S" `jot 16 0` -- Pawel Jakub Dawidek http://www.wheel.pl pjd_at_FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am!
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:20 UTC