Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

Slow Transfer

Pages: 1 2

admin
2,930 Posts
December 20, 2018, 2:11 pm

How many threads did you use in the benchmark?  Can you try 64 ?

Some different sync write ssd speeds

https://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/

Also as stated you can measure the sync write speed from PetaSAN.

#11

Chrisjmuk
17 Posts
December 20, 2018, 2:27 pm

What is the advantage and disadvantage of using a SSD as a journal?

 

here is the results.

 

root@INETC1422:~# fio --filename=/dev/sdb --direct=1 --sync=1 --rw=write --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
journal-test: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
fio-2.2.10
Starting 1 process
Jobs: 1 (f=1): [W(1)] [100.0% done] [0KB/424KB/0KB /s] [0/106/0 iops] [eta 00m:00s]
journal-test: (groupid=0, jobs=1): err= 0: pid=4276: Thu Dec 20 14:25:17 2018
write: io=28224KB, bw=481352B/s, iops=117, runt= 60042msec
clat (msec): min=1, max=132, avg= 8.50, stdev= 4.28
lat (msec): min=1, max=132, avg= 8.50, stdev= 4.28
clat percentiles (usec):
|  1.00th=[ 1544],  5.00th=[ 1704], 10.00th=[ 2480], 20.00th=[ 8032],
| 30.00th=[ 8384], 40.00th=[ 9024], 50.00th=[ 9152], 60.00th=[ 9280],
| 70.00th=[ 9792], 80.00th=[10048], 90.00th=[10816], 95.00th=[11584],
| 99.00th=[14144], 99.50th=[16320], 99.90th=[29824], 99.95th=[130560],
| 99.99th=[132096]
bw (KB  /s): min=  299, max= 1920, per=100.00%, avg=471.52, stdev=276.34
lat (msec) : 2=7.64%, 4=7.64%, 10=61.41%, 20=23.06%, 50=0.18%
lat (msec) : 100=0.01%, 250=0.06%
cpu          : usr=0.16%, sys=0.69%, ctx=14129, majf=0, minf=11
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=0/w=7056/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: io=28224KB, aggrb=470KB/s, minb=470KB/s, maxb=470KB/s, mint=60042msec, maxt=60042msec

Disk stats (read/write):
sdb: ios=70/14110, merge=0/0, ticks=148/58976, in_queue=59144, util=98.35%

#12

Chrisjmuk
17 Posts
December 20, 2018, 2:35 pm

on my second drive i get a huge difference.

 

root@INETC1422:~# fio --filename=/dev/sdc --direct=1 --sync=1 --rw=write --bs=4k --numjobs=4 --iodepth=1 --runtime=60 --time_based --group_reporting --name=journal-test
journal-test: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1
...
fio-2.2.10
Starting 4 processes
Jobs: 4 (f=4): [W(4)] [100.0% done] [0KB/4816KB/0KB /s] [0/1204/0 iops] [eta 00m:00s]
journal-test: (groupid=0, jobs=4): err= 0: pid=5129: Thu Dec 20 14:30:53 2018
write: io=294608KB, bw=4909.1KB/s, iops=1227, runt= 60002msec
clat (msec): min=1, max=129, avg= 3.25, stdev= 2.06
lat (msec): min=1, max=129, avg= 3.25, stdev= 2.06
clat percentiles (msec):
|  1.00th=[    3],  5.00th=[    3], 10.00th=[    4], 20.00th=[    4],
| 30.00th=[    4], 40.00th=[    4], 50.00th=[    4], 60.00th=[    4],
| 70.00th=[    4], 80.00th=[    4], 90.00th=[    4], 95.00th=[    4],
| 99.00th=[    4], 99.50th=[    5], 99.90th=[    9], 99.95th=[   15],
| 99.99th=[  126]
bw (KB  /s): min=  900, max= 1373, per=25.03%, avg=1228.80, stdev=70.82
lat (msec) : 2=0.11%, 4=99.23%, 10=0.58%, 20=0.05%, 250=0.03%
cpu          : usr=0.26%, sys=1.40%, ctx=147454, majf=0, minf=48
IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued    : total=r=0/w=73652/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
WRITE: io=294608KB, aggrb=4909KB/s, minb=4909KB/s, maxb=4909KB/s, mint=60002msec, maxt=60002msec

Disk stats (read/write):
sdc: ios=18/147096, merge=0/0, ticks=16/233388, in_queue=233384, util=99.34%

 

this is my disks

 
Disk /dev/sdc: 953.9 GiB, 1024209543168 bytes, 2000409264 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/sdb: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

 

On my SDB it seems to be difference in I/O size and sectors.

#13

admin
2,930 Posts
December 20, 2018, 9:26 pm

For the PetaSAN benchmark, How many threads did you use in the benchmark?  Can you try 64 ?

For the fio tests, even though you used 4 threads for the second disk and 1 for first, still the second disk is a couple of times faster than first.

#14
Pages: 1 2