Can a 3GB/s HBA such as the LSI Logic SAS3801X PCI-X be used to connect to the J830S?

  • 42 Views
  • Last Post 10 July 2018
Ned Ragdnuos posted this 12 June 2018

I was wondering whether a 3Gb/s HBA card, such as the LSI Logic SAS3801X PCI-X, could be used to connect to a VTrak J830S? I have an old SuperMicro X8DTN+ based system with PCI-X slots (all of its three PCIe solts are occupied with IBM M1015 HBAs for access to its internal drives on a BPN-SAS-846A backplane), so I was seeking a way to copy the data off more quickly by directly-attaching a VTrak J830S to it (rather than using its ethernet connection).

Order By: Standard | Latest | Votes
Pradeep Chidambaranath posted this 13 June 2018

Hi Ned,

I could see that you have created a support case regarding the same questions. Kindly check the support case (Case ID:20180613093437S) for updates.

Regards
Pradeep C

Richard Oettinger posted this 14 June 2018

Hi Ned,

Only the LSI 9200 was tested with the VTrak Jx30s, but the LSI 3801 *might* work - it is a 3Gb SAS HBA with SFF-8088 ports so you could use the SAS cable that comes with the JBOD. If you have one give it a try...

Ned Ragdnuos posted this 04 July 2018

Hi Richard,

Thanks for your advice. I got one of the LSI Logic SAS3801X PCI-X 3Gb/s SAS HBA cards and put it into a SuperMicro with a X8DTN+ motherboard and it worked fine in Debian Jessie (more details on the machine here: http://www.penlug.org/foswiki/bin/view/Main/LinuxHardwareInfoSuperMicroX8DTNplus ) without having to flash any firmware or anything. The card is able to drive twenty-four HUH728080AL5200 drives spread over two J830s enclosures (these drives need the X30SAS12DGL adapter to work in the J830s). The sequential I/O performance for a ZFS setup (see below for the command used to create the ZFS pool) isn't very fast (263MB/s) but it seems adequate for doing "zfs send" of a fairly large filesystem from the SuperMicro machine to the drives in the J830s. Individually, each of the drives are capable of over 200MB/s sequential.

# dd  if=/dev/zero of=./junk bs=1024k count=100000
100000+0 records in
100000+0 records out
104857600000 bytes (105 GB) copied, 397.987 s, 263 MB/s

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
sdz             181.00         0.00     40860.00          0      40860
sdaa            178.00         0.00     33540.00          0      33540
sdab            177.00         0.00     29460.00          0      29460
sdac            183.00         0.00     33296.00          0      33296
sdad            168.00         0.00     23820.00          0      23820
sdae            173.00         0.00     34580.00          0      34580
sdaf            172.00         0.00     37124.00          0      37124
sdag            174.00         0.00     38424.00          0      38424
sdah            170.00         0.00     25488.00          0      25488
sdai            173.00         0.00     26504.00          0      26504
sdaj            176.00         0.00     36120.00          0      36120
sdak            169.00         0.00     30736.00          0      30736
sdal            178.00         0.00     24588.00          0      24588
sdam            176.00         0.00     38164.00          0      38164
sdan            176.00         0.00     38788.00          0      38788
sdao            170.00         0.00     30996.00          0      30996
sdap            180.00         0.00     37144.00          0      37144
sdaq            173.00         0.00     29828.00          0      29828
sdar            170.00         0.00     26384.00          0      26384
sdas            171.00         0.00     38292.00          0      38292
sdat            169.00         0.00     28688.00          0      28688
sdau            176.00         0.00     35092.00          0      35092
sdav            175.00         0.00     41352.00          0      41352
sdaw            169.00         0.00     21904.00          0      21904

The ZFS pool was set up as follows using three-way mirroring to maximize redundancy and read access, so it's not an ideal configuration for the above sequential write experiment, but at least it gives an idea of what the card can do.

/sbin/modprobe zfs

sudo zpool create -f -o ashift=12 promise                     \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy0-lun-0  \
       pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy16-lun-0 \
       pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy8-lun-0  \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy1-lun-0  \
       pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy17-lun-0 \
       pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy9-lun-0  \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy2-lun-0  \
       pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy18-lun-0 \
       pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy10-lun-0 \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy3-lun-0  \
       pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy19-lun-0 \
       pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy11-lun-0 \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy8-lun-0  \
       pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy0-lun-0  \
       pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy16-lun-0 \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy9-lun-0  \
       pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy1-lun-0  \
       pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy17-lun-0 \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy10-lun-0 \
       pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy2-lun-0  \
       pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy18-lun-0 \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy11-lun-0 \
       pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy3-lun-0  \
       pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy19-lun-0

Ned Ragdnuos posted this 04 July 2018

The cables connecting the SAS3801X card to the two J830s are 10 meter long (32 foot) EMC Amphenol 038-003-658 cables (with the annoying little anti-insertion pieces cut off at the SAS3801X end of the cables). Using such long cables might be slowing things down a little.

Richard Oettinger posted this 05 July 2018

Hi Ned,

that might be a problem, I believe the max length supported is 5 meters.

One way to check is to set the SAS driver value on the JBOD.

Use the VTrak serial cable and a terminal emulator to access the VTrak IO Module CLI:

cli> cable

CN#1 Cable Length = 1 meter

CN#2 Cable Length = 1 meter

CN#1 is the SAS OUT port and CN#2 is the SAS IN port, so you want to set the SAS IN port length:

cli> cable -a mod -s "cn2=5"

Verify the change:

cli> cable

CN#1 Cable Length = 1 meter

CN#2 Cable Length = 5 meter

Connect the cable to the other VTrak IO Module and do the same thing.

* You could try using 10 meters for the cable value and see it it accepts it... *

 

Ned Ragdnuos posted this 10 July 2018

Hi Richard,

Thanks for your advice.

I hadn't come across the "cable" command of the cli in the manual I was looking at.

Could I be looking in the wrong manual? Is there another manual that covers all of the cli commands, or am I looking in the wrong section?

I found that the command you suggested does accept 8 meter as the length. I tried that with my ten meter cable but then I started getting a bunch of scsi errors writing to the pool. After rebooting and going back to the initial setting of one meter, I don't get any scsi errors.

Do I need to restart the J830s for the cable length setting to take effect?

I'll get an actual 8 meter cable and see if it helps.

In the meantime, I got another of the LSI Logic SAS3801X PCI-X 3Gb/s SAS HBA cards and connected the second J830s to the second card (hoping to spread the I/O between them). It seems to be helping speed things up, but I haven't done proper measurements yet.

 

Close