Individual disk write rate vs. same disk write in a RAID0 configuration

  • 110 Views
  • Last Post 17 March 2017
Barry Sharp posted this 05 March 2017

I have a Pegasus2 R8 with 8 4TB disks. Each disk is a Hitachi HGST HDN724040AL that spins at 7200 RPM.

The R8 is connected directly to my MP6,1 that has Thunderbolt 2 ports.

When I configure the R8 as RAID0 across all 8 disks and using the Promise Utility to observe the individual disk write rates I note the display indicates each disk is running at around 116 MBytes/sec. This is providing some 950 MBytes/sec.

If I remove one of the Hitachi 4TB disks from the R8 and place it in a 3.5" TB2 enclosure and run a dd command to write data to it the write rate I observe is around 165 MBytes/sec.

In each case the disk(s) have 100% free space when I test.

If the R8 in a RAID0 across all 8 disks were to provide 165 MBytes/sec each then I would expect to observe some 1320 MBytes/sec.

Question:

Why the difference between the 950 MB/s vs. the 1320 MB/s I'm seeing ?

There seems to me there's a significant overhead in the R8 when configured as RAID0 across the 8 disks.

Maybe the RAID0 Logical Drive settings need to be tweak to improve its write performance.

Is there advice on how best to set the options provided by the Promise Utility for improving the write speed ?

The test I use for the R8 configured as RAID0 over 8 disks is...

dd bs=10000000 count=10000000 if=/dev/zero of=/Volumes/P_Stripe/test_file

The test I use when testing the single disk write rate is...

dd bs=10000000 count=10000000 if=/dev/zero of=/Volumes/HD/test_file

Thank you for any advice yo can provide, and maybe an explanation for why I'm seeing the difference of 950 MB/s vs. the potential 1320 MB/s.

Order By: Standard | Latest | Votes
Pradeep Chidambaranath posted this 05 March 2017

Hi Barry,

 

Please check the below KB article to know more about the write speed on a Pegasus2 R8 unit.

http://kb.promise.com/thread/how-to-bench-mark-pegasus2-r8/

Kindly, go to https://support.promise.com to create a support ticket with the subsystem report for further investigation.

Follow the below steps to save the subsystem report.

- Open Promise Utility.
- Click subsystem information icon on the top of the window.
- Click the lock symbol on the left bottom corner of the screen to unlock the utility. - Click on save service report button to save the subsystem report.  

Regards

Pradeep C

Barry Sharp posted this 05 March 2017

The "Forced ReadAhead: Enabled" is set.

Barry Sharp posted this 05 March 2017

 When I execute the promiseutil I get this (note: I have 3 Pegasus enclosures; Pegasus R4, Pegasus2 R6 and Pegasus2 R8)

Zeus-Mac-Pro:~ bxs$ promiseutil

-------------------------------------------------------------

Promise Utility

Version: 3.20.0000.15 Build Date: Sep 29, 2014

-------------------------------------------------------------

 

List available RAID HBAs and Subsystems

===============================================================================

Type  #    Model         Alias                         WWN                 Seq

===============================================================================

hba   1  * Pegasus R4                                  2000-0001-5554-9b1e  3 

hba   2    Pegasus2 R6                                 2000-0001-5557-5b1d  2 

hba   3    Pegasus2 R8   R8_Controller                 2000-0001-5558-7c20  1 

 

Totally 3 HBA(s) and 0 Subsystem(s)

 

-------------------------------------------------------------

The row with '*' sign refers the current working HBA/Subsystem path

To change the current HBA/Subsystem path, you may use the following command:

  

  spath -a chgpath -t hba|subsys -p <path #>.

 

Type help or ? to display all the available commands

-------------------------------------------------------------

The help "spath -a chgpath -t hba|subsys -p <path #>" command does not seem to work for me. I've tried various combinations of this command but they all issue an error. For example how to I change the HBA/Subsystem path to be for the "hba 3 Pegasus2 R8" ?

Here's what I've tried... and after a while I simply disconnected the R4 and R6 so that only the R8 remained so I could obtain the output of ctrl -v which is shown below as my last attempt. You will see that the "ForcedReadAhead: Enabled" is set for the R8.

So what additional advice can you provide to setup things to be able to get the performance up to around 1350 MB/s for the R8 configured as RAID0 ? Thank you.

cliib> spath -a chgpath -t hba -p 3

cliib> spath

===============================================================================

Type  #    Model         Alias                         WWN                 Seq

===============================================================================

hba   1  * Pegasus R4                                  2000-0001-5554-9b1e  3 

hba   2    Pegasus2 R6                                 2000-0001-5557-5b1d  2 

hba   3    Pegasus2 R8   R8_Controller                 2000-0001-5558-7c20  1 

Totally 3 HBA(s) and 0 Subsystem(s)

cliib> spath -a chgpath -t hba -p 1

 

cliib> spath

===============================================================================

Type  #    Model         Alias                         WWN                 Seq

===============================================================================

hba   1  * Pegasus R4                                  2000-0001-5554-9b1e  3 

hba   2    Pegasus2 R6                                 2000-0001-5557-5b1d  2 

hba   3    Pegasus2 R8   R8_Controller                 2000-0001-5558-7c20  1 

 

Totally 3 HBA(s) and 0 Subsystem(s)

 

cliib> spath -a chgpath -t hba -p3

 

cliib> spath

===============================================================================

Type  #    Model         Alias                         WWN                 Seq

===============================================================================

hba   1  * Pegasus R4                                  2000-0001-5554-9b1e  3 

hba   2    Pegasus2 R6                                 2000-0001-5557-5b1d  2 

hba   3    Pegasus2 R8   R8_Controller                 2000-0001-5558-7c20  1 

 

Totally 3 HBA(s) and 0 Subsystem(s)

cliib> ctrl -v

 

-------------------------------------------------------------------------------

CtrlId: 1

Alias: 

OperationalStatus: OK                  PowerOnTime: 13 hours 46 minutes

LUNAffinity: N/A                       LunmappingMethod: WWN Based

CacheUsagePercentage: 0%               DirtyCachePercentage: 0%

 

Vendor: PROMISE                        Model: Pegasus R4

PartNo: F29DS4722000000                SerialNo: M74H12923400161

HWRev: A2                              WWN: 2000-0001-5554-9b1e

CmdProtocol: SCSI-3

MemType: DDR2 SDRAM                    MemSize: 512MB

FlashType: Flash Memory                FlashSize: 8MB

NVRAMType: FRAM                        NVRAMSize: 32KB

BootLoaderVersion: 5.04.0000.57        BootLoaderBuildDate: Dec 28, 2015

FirmwareVersion: 5.04.0000.57          FirmwareBuildDate: Dec 28, 2015

SoftwareVersion: 5.04.0000.57          SoftwareBuildDate: Dec 28, 2015

BIOSVersion: 5.04.0000.57              BIOSBuildDate: Dec 28, 2015

SingleImageVersion: 5.04.0000.57       SingleImageBuildDate: Dec 28, 2015

CtrlCPLDVersion: 0.08

 

DiskArrayPresent: 1                    OverallRAIDStatus: OK

LogDrvPresent: 3                       LogDrvOnline: 3

LogDrvOffline: 0                       LogDrvCritical: 0

PhyDrvPresent: 4                       PhyDrvOnline: 4

PhyDrvOffline: 0                       PhyDrvPFA: 0

GlobalSparePresent: 0                  DedicatedSparePresent: 0

RevertibleGlobalSparePresent: 0        RevertibleDedicatedSparePresent: 0

RevertibleGlobalSpareUsed: 0           RevertibleDedicatedSpareUsed: 0

 

WriteThroughMode: No                   MaxSectorSize: 4KB

PreferredCacheLineSize: 64 KB          CacheLineSize: 64 KB

Coercion: Enabled                      CoercionMethod: GBTruncate

SMART: Enabled                         SMARTPollingInterval: 58017211 minutes

MigrationStorage: DDF                  CacheFlushInterval: 3 second(s)

PollInterval: 15 second(s)             AdaptiveWBCache: Disabled

ForcedReadAhead: Enabled

 

PowerSavingLevel: 0                    SpindownType: N/A

IdleTimeToParkRwHeads: Never           IdleTimeToLowerRotationSpeed: Never

IdleTimeToSpinDown: Never

 

SGPIOBackPlane: Default/Generic (0)    SASReadyLED: Off

SpinUpDelay: 0 millisecond             PowerManagement: Enable

 

SecurityAdminPassword: N/A

 

PciFuncAddr: N/A                       PciLinkSpeed: N/A

PciLinkWidth: N/A

 

cliib>

 

Zeus-Mac-Pro:~ bxs$ promiseutil

-------------------------------------------------------------

Promise Utility

Version: 3.20.0000.15 Build Date: Sep 29, 2014

-------------------------------------------------------------

 

List available RAID HBAs and Subsystems

===============================================================================

Type  #    Model         Alias                         WWN                 Seq

===============================================================================

hba   1  * Pegasus2 R8   R8_Controller                 2000-0001-5558-7c20  1 

 

Totally 1 HBA(s) and 0 Subsystem(s)

 

-------------------------------------------------------------

The row with '*' sign refers the current working HBA/Subsystem path

To change the current HBA/Subsystem path, you may use the following command:

  

  spath -a chgpath -t hba|subsys -p <path #>.

 

Type help or ? to display all the available commands

-------------------------------------------------------------

 

cliib> ctrl -v

 

-------------------------------------------------------------------------------

CtrlId: 1

Alias: R8_Controller

OperationalStatus: OK, BGA Running     PowerOnTime: 2 minutes

LUNAffinity: N/A                       LunmappingMethod: WWN Based

CacheUsagePercentage: 0%               DirtyCachePercentage: 0%

 

Vendor: PROMISE                        Model: Pegasus2 R8

PartNo: F29DS8722000000                SerialNo: M92L14408000146

HWRev: B3                              WWN: 2000-0001-5558-7c20

CmdProtocol: SCSI-3

MemType: DDR2 SDRAM                    MemSize: 512MB

FlashType: Flash Memory                FlashSize: 8MB

NVRAMType: FRAM                        NVRAMSize: 128KB

BootLoaderVersion: 5.04.0000.57        BootLoaderBuildDate: Dec 28, 2015

FirmwareVersion: 5.04.0000.57          FirmwareBuildDate: Dec 28, 2015

SoftwareVersion: 5.04.0000.57          SoftwareBuildDate: Dec 28, 2015

BIOSVersion: 5.04.0000.57              BIOSBuildDate: Dec 28, 2015

SingleImageVersion: 5.04.0000.57       SingleImageBuildDate: Dec 28, 2015

CtrlCPLDVersion: 6.08

 

DiskArrayPresent: 1                    OverallRAIDStatus: OK

LogDrvPresent: 6                       LogDrvOnline: 6

LogDrvOffline: 0                       LogDrvCritical: 0

PhyDrvPresent: 8                       PhyDrvOnline: 8

PhyDrvOffline: 0                       PhyDrvPFA: 0

GlobalSparePresent: 0                  DedicatedSparePresent: 0

RevertibleGlobalSparePresent: 0        RevertibleDedicatedSparePresent: 0

RevertibleGlobalSpareUsed: 0           RevertibleDedicatedSpareUsed: 0

 

WriteThroughMode: No                   MaxSectorSize: 4KB

PreferredCacheLineSize: 64 KB          CacheLineSize: 64 KB

Coercion: Enabled                      CoercionMethod: GBTruncate

SMART: Enabled                         SMARTPollingInterval: 10 minutes

MigrationStorage: DDF                  CacheFlushInterval: 3 second(s)

PollInterval: 15 second(s)             AdaptiveWBCache: Disabled

ForcedReadAhead: Enabled

 

PowerSavingLevel: 0                    SpindownType: All drives

IdleTimeToParkRwHeads: Never           IdleTimeToLowerRotationSpeed: Never

IdleTimeToSpinDown: Never

 

SGPIOBackPlane: Default/Generic (0)    SASReadyLED: Off

SpinUpDelay: 0 millisecond             PowerManagement: Enable

 

SecurityAdminPassword: N/A

 

PciFuncAddr: N/A                       PciLinkSpeed: N/A

PciLinkWidth: N/A

 

cliib>

Devendra Kumar posted this 05 March 2017

Hi Barry,

Please find the below command to switch to the different subsystem.

spath -a chgpath -t hba -p 2  

(P- refers to subsystem order).

Please refer the below KB article

http://kb.promise.com/search/?term=spath

Thanks

Richard Oettinger posted this 05 March 2017

If I may add my two cents here:

The total maximum write speed of a RAID system is not the product of the max write speed of a disk times the number of disks used. The RAID controller does have some overhead, but an important consideration - among a few others - is that the sectors of all the drives are not synchronized, which means that when one drive is writing its chunk of data, the others have to wait for the disk to rotate to the start of the requried sector. This tends to average out to the speed you are seeing.

If you were to put SSDs in to your Pegasus you would see around 1300MB/s write speeds as they do not have any rotational latency...

~ Richard

Barry Sharp posted this 05 March 2017

Richard: Thanks for that explanation. It makes sense to me.

 

BTW.... Where would I find SSD sleds that would be compatible with the Pegasus enclosure ?

Barry Sharp posted this 06 March 2017

Devendra Kumar wrote.....

 

Hi Barry,

Please find the below command to switch to the different subsystem.

spath -a chgpath -t hba -p 2  

(P- refers to subsystem order).

Please refer the below KB article

http://kb.promise.com/search/?term=spath

Thanks

 

Note: I find that to switch to a different subsystem the promiseutil must be executed with root priveleges. The KB article does not mention this as far as I can see. If I'm correct, then the KB article needs to be corrected/revised. Thank you.

The promiseutil needs to be launched in Terminal as....

sudo promiseutil

Richard Oettinger posted this 17 March 2017

BTW.... Where would I find SSD sleds that would be compatible with the Pegasus enclosure ?

 

Barry, I have not seen empty sleds for sale, but since you have to take the HDDs out, just un-screw the drives from the trays you have and screw the SSDs back on. There are mounting holes for 2.5" drives... 

Search
Categories
Close