I was wondering whether a 3Gb/s HBA card, such as the LSI Logic SAS3801X PCI-X, could be used to connect to a VTrak J830S? I have an old SuperMicro X8DTN+ based system with PCI-X slots (all of its three PCIe solts are occupied with IBM M1015 HBAs for access to its internal drives on a BPN-SAS-846A backplane), so I was seeking a way to copy the data off more quickly by directly-attaching a VTrak J830S to it (rather than using its ethernet connection).
Can a 3GB/s HBA such as the LSI Logic SAS3801X PCI-X be used to connect to the J830S?
- 1.2K Views
- Last Post 10 July 2018
Ned Ragdnuos
posted this
12 June 2018
- Last edited 12 June 2018
Pradeep Chidambaranath
posted this
13 June 2018
Hi Ned,
I could see that you have created a support case regarding the same questions. Kindly check the support case (Case ID:20180613093437S) for updates.
Regards
Pradeep C
Richard Oettinger
posted this
14 June 2018
Hi Ned,
Only the LSI 9200 was tested with the VTrak Jx30s, but the LSI 3801 *might* work - it is a 3Gb SAS HBA with SFF-8088 ports so you could use the SAS cable that comes with the JBOD. If you have one give it a try...
Ned Ragdnuos
posted this
04 July 2018
Hi Richard,
Thanks for your advice. I got one of the LSI Logic SAS3801X PCI-X 3Gb/s SAS HBA cards and put it into a SuperMicro with a X8DTN+ motherboard and it worked fine in Debian Jessie (more details on the machine here: http://www.penlug.org/foswiki/bin/view/Main/LinuxHardwareInfoSuperMicroX8DTNplus ) without having to flash any firmware or anything. The card is able to drive twenty-four HUH728080AL5200 drives spread over two J830s enclosures (these drives need the X30SAS12DGL adapter to work in the J830s). The sequential I/O performance for a ZFS setup (see below for the command used to create the ZFS pool) isn't very fast (263MB/s) but it seems adequate for doing "zfs send" of a fairly large filesystem from the SuperMicro machine to the drives in the J830s. Individually, each of the drives are capable of over 200MB/s sequential.
# dd if=/dev/zero of=./junk bs=1024k count=100000
100000+0 records in
100000+0 records out
104857600000 bytes (105 GB) copied, 397.987 s, 263 MB/s
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sdz 181.00 0.00 40860.00 0 40860
sdaa 178.00 0.00 33540.00 0 33540
sdab 177.00 0.00 29460.00 0 29460
sdac 183.00 0.00 33296.00 0 33296
sdad 168.00 0.00 23820.00 0 23820
sdae 173.00 0.00 34580.00 0 34580
sdaf 172.00 0.00 37124.00 0 37124
sdag 174.00 0.00 38424.00 0 38424
sdah 170.00 0.00 25488.00 0 25488
sdai 173.00 0.00 26504.00 0 26504
sdaj 176.00 0.00 36120.00 0 36120
sdak 169.00 0.00 30736.00 0 30736
sdal 178.00 0.00 24588.00 0 24588
sdam 176.00 0.00 38164.00 0 38164
sdan 176.00 0.00 38788.00 0 38788
sdao 170.00 0.00 30996.00 0 30996
sdap 180.00 0.00 37144.00 0 37144
sdaq 173.00 0.00 29828.00 0 29828
sdar 170.00 0.00 26384.00 0 26384
sdas 171.00 0.00 38292.00 0 38292
sdat 169.00 0.00 28688.00 0 28688
sdau 176.00 0.00 35092.00 0 35092
sdav 175.00 0.00 41352.00 0 41352
sdaw 169.00 0.00 21904.00 0 21904
The ZFS pool was set up as follows using three-way mirroring to maximize redundancy and read access, so it's not an ideal configuration for the above sequential write experiment, but at least it gives an idea of what the card can do.
/sbin/modprobe zfs
sudo zpool create -f -o ashift=12 promise \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy0-lun-0 \
pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy16-lun-0 \
pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy8-lun-0 \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy1-lun-0 \
pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy17-lun-0 \
pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy9-lun-0 \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy2-lun-0 \
pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy18-lun-0 \
pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy10-lun-0 \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy3-lun-0 \
pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy19-lun-0 \
pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy11-lun-0 \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy8-lun-0 \
pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy0-lun-0 \
pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy16-lun-0 \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy9-lun-0 \
pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy1-lun-0 \
pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy17-lun-0 \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy10-lun-0 \
pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy2-lun-0 \
pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy18-lun-0 \
mirror pci-0000:03:00.0-sas-exp0x5000155585afc23f-phy11-lun-0 \
pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy3-lun-0 \
pci-0000:04:00.0-sas-exp0x5000155585d4e23f-phy19-lun-0
Ned Ragdnuos
posted this
04 July 2018
The cables connecting the SAS3801X card to the two J830s are 10 meter long (32 foot) EMC Amphenol 038-003-658 cables (with the annoying little anti-insertion pieces cut off at the SAS3801X end of the cables). Using such long cables might be slowing things down a little.
Richard Oettinger
posted this
05 July 2018
Hi Ned,
that might be a problem, I believe the max length supported is 5 meters.
One way to check is to set the SAS driver value on the JBOD.
Use the VTrak serial cable and a terminal emulator to access the VTrak IO Module CLI:
cli> cable
CN#1 Cable Length = 1 meter
CN#2 Cable Length = 1 meter
CN#1 is the SAS OUT port and CN#2 is the SAS IN port, so you want to set the SAS IN port length:
cli> cable -a mod -s "cn2=5"
Verify the change:
cli> cable
CN#1 Cable Length = 1 meter
CN#2 Cable Length = 5 meter
Connect the cable to the other VTrak IO Module and do the same thing.
* You could try using 10 meters for the cable value and see it it accepts it... *
Ned Ragdnuos
posted this
10 July 2018
Hi Richard,
Thanks for your advice.
I hadn't come across the "cable" command of the cli in the manual I was looking at.
Could I be looking in the wrong manual? Is there another manual that covers all of the cli commands, or am I looking in the wrong section?
I found that the command you suggested does accept 8 meter as the length. I tried that with my ten meter cable but then I started getting a bunch of scsi errors writing to the pool. After rebooting and going back to the initial setting of one meter, I don't get any scsi errors.
Do I need to restart the J830s for the cable length setting to take effect?
I'll get an actual 8 meter cable and see if it helps.
In the meantime, I got another of the LSI Logic SAS3801X PCI-X 3Gb/s SAS HBA cards and connected the second J830s to the second card (hoping to spread the I/O between them). It seems to be helping speed things up, but I haven't done proper measurements yet.
