Re: PCIe NVME drives not detected on Dell R6515

From: Scott Long <scottl_at_samsco.org>
Date: Fri, 17 Apr 2020 10:17:00 -0600
You are correct about Intel vs AMD.  Comparing the full output of pciconf from FreeBSD with the fragment of lspci from Linux suggests that there’s at least one set of a PCIe switch and child devices that is not being enumerated by FreeBSD.  Can you send the full output of `lspci -tvv` from linux?

Thanks,
Scott


> On Apr 17, 2020, at 9:54 AM, Scott Long <scottl_at_samsco.org> wrote:
> 
> Would that be the Intel VMD/VROC stuff?  If so, there’s a driver for FreeBSD, but it’s not well tested yet.  Will have to dig in further.
> 
> Scott
> 
> 
>> On Apr 17, 2020, at 9:50 AM, Warner Losh <imp_at_bsdimp.com> wrote:
>> 
>> 
>> 
>> On Fri, Apr 17, 2020 at 9:39 AM Scott Long <scottl_at_samsco.org> wrote:
>> Can you send me the output of ‘pciconf -llv’, either in 12-STABLE or 13-CURRENT?  Also, can you send me the output of ‘dmesg’?
>> 
>> There was another thread that said there was a raid card in the way... It would be cool to find a way to get it out of the way... :)
>> 
>> Warner
>> 
>> Thanks,
>> Scott
>> 
>> 
>>> On Apr 17, 2020, at 5:23 AM, Miroslav Lachman <000.fbsd_at_quip.cz> wrote:
>>> 
>>> I already asked on stable_at_ but as I tried it on 13-CURRENT with the same result I am trying to ask for help here.
>>> 
>>> I have rented dedicated server Dell PowerEdge R6515 with iDRAC access only.
>>> There are 2 NVME drives which I wanted to use as ZFS root pool.
>>> 
>>> They are shown in iDRAC
>>> 
>>> Device Description:   PCIe SSD in Slot 1 in Bay 1
>>> Device Protocol:      NVMe-MI1.0
>>> Model:                Dell Express Flash NVMe P4510 1TB SFF
>>> Bus:                  130
>>> Manufacturer:         INTEL
>>> Product ID:           a54
>>> Revision:             VDV1DP23
>>> Enclosure:            PCIe SSD Backplane 1
>>> 
>>> 
>>> pciconf -l show many things, some of them are named "noneN_at_pci..." but none "nvme"
>>> 
>>> The is printscreen (12.1 but 13-CURRENT is the same)
>>> 
>>> https://ibb.co/tPnymL7
>>> 
>>> But I booted Linux SystemRescueCd and nvme devices are there visible in /dev/
>>> https://ibb.co/sj22Nwg
>>> 
>>> There is verbose output of Linux lspci https://ibb.co/dPZTwV1
>>> 
>>> Linux dmesg contains:
>>> nvme nvme0: pci function 0000:81:00.0
>>> nvme nvme1: pci function 0000:82:00.0
>>> nvme nvme0: 32/0/0 default/read/poll queues
>>> nvme nvme1: 32/0/0 default/read/poll queues
>>> 
>>> 
>>> The machine is Dell PowerEdge R6515 with AMD EPYC 7302P
>>> 
>>> 
>>> More details extracted from iDRAC web interface
>>> 
>>> I found this informations
>>> 
>>> PCIe SSD in Slot 1 in Bay 1
>>> Bus:   82
>>> BusProtocol:  PCIE
>>> Device:  0
>>> DeviceDescription:  PCIe SSD in Slot 1 in Bay 1
>>> DeviceProtocol:  NVMe-MI1.0
>>> DeviceType:  PCIeSSD
>>> DriveFormFactor:  2.5 inch
>>> FailurePredicted:  NO
>>> FQDD:  Disk.Bay.1:Enclosure.Internal.0-1
>>> FreeSizeInBytes:  Information Not Available
>>> Function:  0
>>> HotSpareStatus:  Information Not Available
>>> InstanceID:  Disk.Bay.1:Enclosure.Internal.0-1
>>> Manufacturer:  INTEL
>>> MaximumCapableSpeed:  8 GT/s
>>> MediaType:  Solid State Drive
>>> Model:  Dell Express Flash NVMe P4510 1TB SFF
>>> NegotiatedSpeed:  8 GT/s
>>> PCIeCapableLinkWidth:  x4
>>> PCIeNegotiatedLinkWidth:  x4
>>> PrimaryStatus:  Ok
>>> ProductID:  a54
>>> RaidStatus:  Information Not Available
>>> RAIDType:  Unknown
>>> RemainingRatedWriteEndurance:  100 %
>>> Revision:  VDV1DP23
>>> SerialNumber:  PHLJxxxxxxWF1PxxxxN
>>> SizeInBytes:  1000204886016
>>> Slot:  1
>>> State:  Ready
>>> SystemEraseCapability:  CryptographicErasePD
>>> 
>>> PCIe SSD in Slot 1 in Bay 1 - PCI Device
>>> BusNumber:  130
>>> DataBusWidth:  4x or x4
>>> Description:  Express Flash NVMe 1.0 TB 2.5" U.2 (P4510)
>>> DeviceDescription:  PCIe SSD in Slot 1 in Bay 1
>>> DeviceNumber:  0
>>> DeviceType:  PCIDevice
>>> FQDD:  Disk.Bay.1:Enclosure.Internal.0-1
>>> FunctionNumber:  0
>>> InstanceID:  Disk.Bay.1:Enclosure.Internal.0-1
>>> LastSystemInventoryTime:  2020-04-17T03:21:39
>>> LastUpdateTime:  2020-03-31T13:55:06
>>> Manufacturer:  Intel Corporation
>>> PCIDeviceID:  0A54
>>> PCISubDeviceID:  2003
>>> PCISubVendorID:  1028
>>> PCIVendorID:  8086
>>> SlotLength:  2.5 Inch Drive Form Factor
>>> SlotType:  PCI Express Gen 3 SFF-8639
>>> 
>>> 
>>> Can anybody shed some light what the real problem is?
>>> 
>>> Is the hardware not properly detected or is the driver completely missing?
>>> 
>>> NVME PCIe architecture is out of my knowledge.
>>> 
>>> I really appreciate any help.
>>> 
>>> Kind regards
>>> Miroslav Lachman
>>> _______________________________________________
>>> freebsd-current_at_freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
>> 
>> _______________________________________________
>> freebsd-current_at_freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
> 
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
Received on Fri Apr 17 2020 - 14:17:02 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:23 UTC