CORAID speed test

Hardware: 2x10Gb link, 36x2TB drives on CORAID SRX 4200.

aoe-stat
e0.X 2000.398GB eth2,eth3 8704 up
(36 drives total)
ethtool eth2
Speed: 10000Mb/s
ethtool eth3
Speed: 10000Mb/s
 
cec -s 0 eth2
disks
0.0 2000.398GB X.0.0 WDC WD2003FYYS-02W0B0 01.01D01 sata 3.0Gb/s
(36 drives total)
list -l
X 2000.399GB online
X.0 2000.399GB jbod normal
X.0.0 normal 2000.399GB 0.X
(36 JBODs total)

CORAID raw device speed (using aio, O_DIRECT to omit the cache, O_SYNC and different RAID types):

• JBODs, 36 LUNs per 2TB, 4 simultaneous request/LUN, read and write by 4K blocks

50% reads, 50% writes = 220.45 iops/LUN, 2m5.678s/1000000 blocks
reads = 163.39 iops/LUN, 2m49.834s/1000000 blocks
writes = 455.37 iops/LUN, 1m0.602s/1000000 blocks

• raid6rs, 4 LUNs per 14TB, 100 simultaneous request/LUN, read and write by 4K blocks

50% reads, 50% writes = 274.72 iops/LUN, 1m31.482s/100000 blocks
reads = 1524.39 iops/LUN, 2m43.648s/1000000 blocks
writes = 164.47 iops/LUN, 2m31.699s/100000 blocks

• raid5, 4 LUNs per 16TB, 100 simultaneous request/LUN, read and write by 4K blocks

50% reads, 50% writes = 827.81 iops/LUN, 2m31.095s/500000 blocks
reads = 1519.94 iops/LUN, 2m44.482s/1000000 blocks
writes = 555.55 iops/LUN, 0m44.665s/100000 blocks

See the test program. You need libaio for it (POSIX aio in librt uses pthreads which is not appropriate because of massive parallelism in our tests).