randrw: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64 ... randrw: (g=0): rw=randrw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=64 Starting 16 processes randrw: (groupid=0, jobs=1): err= 0: pid=2547 read : io=6,124MB, bw=66,100KB/s, iops=16,524, runt= 94870msec slat (usec): min=12, max=27,971, avg=174.21, stdev=266.86 clat (usec): min=3, max=864K, avg=2106.47, stdev=1738.44 bw (KB/s) : min= 0, max=107584, per=0.61%, avg=5409.05, stdev=19273.52 write: io=2,044MB, bw=22,063KB/s, iops=5,515, runt= 94870msec slat (usec): min=13, max=27,970, avg=176.52, stdev=272.77 clat (usec): min=4, max=53,769, avg=2105.06, stdev=1560.23 bw (KB/s) : min= 0, max=35984, per=1.15%, avg=3379.14, stdev=8489.10 cpu : usr=2.53%, sys=94.42%, ctx=6501, majf=0, minf=356 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1567718/523290, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 50=0.01%, 100=0.01%, 250=0.01% lat (usec): 500=0.07%, 750=0.44%, 1000=4.52% lat (msec): 2=60.94%, 4=27.23%, 10=6.10%, 20=0.62%, 50=0.05% lat (msec): 100=0.01%, 500=0.01%, 1000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2550 read : io=6,128MB, bw=58,320KB/s, iops=14,580, runt=107600msec slat (usec): min=12, max=24,164, avg=196.17, stdev=275.12 clat (usec): min=3, max=935K, avg=2393.34, stdev=1837.28 bw (KB/s) : min= 0, max=170576, per=0.42%, avg=3737.76, stdev=15458.85 write: io=2,040MB, bw=19,413KB/s, iops=4,853, runt=107600msec slat (usec): min=11, max=24,162, avg=197.17, stdev=256.00 clat (usec): min=4, max=312K, avg=2391.86, stdev=1708.51 bw (KB/s) : min= 0, max=55960, per=0.99%, avg=2897.19, stdev=7540.20 cpu : usr=2.92%, sys=94.04%, ctx=11358, majf=0, minf=367 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1568810/522198, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01% lat (usec): 250=0.03%, 500=0.35%, 750=1.72%, 1000=4.74% lat (msec): 2=45.40%, 4=37.23%, 10=9.87%, 20=0.63%, 50=0.04% lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 1000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2552 read : io=6,126MB, bw=57,739KB/s, iops=14,434, runt=108647msec slat (usec): min=11, max=22,024, avg=183.39, stdev=275.04 clat (usec): min=2, max=8,027K, avg=2495.05, stdev=45792.61 bw (KB/s) : min= 0, max=183488, per=0.49%, avg=4288.32, stdev=17503.79 write: io=2,042MB, bw=19,245KB/s, iops=4,811, runt=108647msec slat (usec): min=12, max=22,023, avg=185.99, stdev=291.58 clat (usec): min=3, max=8,027K, avg=2431.55, stdev=40055.79 bw (KB/s) : min= 0, max=61248, per=1.08%, avg=3162.73, stdev=8362.27 cpu : usr=2.37%, sys=87.04%, ctx=14078, majf=0, minf=354 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1568288/522720, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01% lat (usec): 250=0.03%, 500=0.61%, 750=2.93%, 1000=6.95% lat (msec): 2=52.36%, 4=26.64%, 10=9.74%, 20=0.68%, 50=0.05% lat (msec): 100=0.01%, 250=0.01%, >=2000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2553 read : io=6,128MB, bw=68,578KB/s, iops=17,144, runt= 91509msec slat (usec): min=12, max=21,715, avg=167.06, stdev=193.91 clat (usec): min=3, max=987K, avg=2033.80, stdev=3616.51 bw (KB/s) : min= 0, max=99856, per=0.61%, avg=5392.37, stdev=19330.77 write: io=2,040MB, bw=22,824KB/s, iops=5,705, runt= 91509msec slat (usec): min=12, max=15,149, avg=168.73, stdev=188.41 clat (usec): min=3, max=610K, avg=2040.64, stdev=4036.89 bw (KB/s) : min= 0, max=33720, per=1.21%, avg=3563.95, stdev=8712.34 cpu : usr=2.40%, sys=94.98%, ctx=6749, majf=0, minf=294 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1568867/522141, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 50=0.01%, 100=0.01%, 250=0.01% lat (usec): 500=0.03%, 750=0.26%, 1000=3.95% lat (msec): 2=62.25%, 4=28.61%, 10=4.59%, 20=0.29%, 50=0.01% lat (msec): 100=0.01%, 750=0.01%, 1000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2554 read : io=6,129MB, bw=57,566KB/s, iops=14,391, runt=109021msec slat (usec): min=11, max=868K, avg=198.80, stdev=1023.29 clat (usec): min=4, max=947K, avg=2426.93, stdev=5200.69 bw (KB/s) : min= 0, max=149128, per=0.50%, avg=4440.55, stdev=17256.87 write: io=2,039MB, bw=19,153KB/s, iops=4,788, runt=109021msec slat (usec): min=11, max=868K, avg=203.19, stdev=1725.25 clat (usec): min=3, max=870K, avg=2417.48, stdev=4278.32 bw (KB/s) : min= 0, max=49472, per=1.01%, avg=2973.29, stdev=7867.38 cpu : usr=2.87%, sys=93.23%, ctx=12934, majf=0, minf=363 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1568981/522027, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 50=0.01%, 100=0.01%, 250=0.02% lat (usec): 500=0.44%, 750=2.30%, 1000=5.96% lat (msec): 2=52.55%, 4=24.84%, 10=12.87%, 20=0.96%, 50=0.06% lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 1000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2555 read : io=6,126MB, bw=59,703KB/s, iops=14,925, runt=105069msec slat (usec): min=11, max=27,921, avg=186.30, stdev=275.04 clat (usec): min=2, max=1,007K, avg=2352.14, stdev=9607.09 bw (KB/s) : min= 0, max=111976, per=0.46%, avg=4078.26, stdev=16419.93 write: io=2,042MB, bw=19,902KB/s, iops=4,975, runt=105069msec slat (usec): min=12, max=24,662, avg=188.34, stdev=273.10 clat (usec): min=3, max=1,007K, avg=2349.94, stdev=9357.34 bw (KB/s) : min= 0, max=37288, per=1.03%, avg=3016.35, stdev=7836.94 cpu : usr=2.41%, sys=91.31%, ctx=8846, majf=0, minf=362 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1568232/522776, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01% lat (usec): 250=0.02%, 500=0.13%, 750=0.64%, 1000=4.32% lat (msec): 2=55.52%, 4=29.73%, 10=9.11%, 20=0.47%, 50=0.05% lat (msec): 100=0.01%, 750=0.01%, 1000=0.01%, 2000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2559 read : io=6,124MB, bw=62,863KB/s, iops=15,715, runt= 99763msec slat (usec): min=9, max=23,329, avg=182.81, stdev=262.64 clat (usec): min=3, max=334K, avg=2216.18, stdev=1519.34 bw (KB/s) : min= 0, max=109664, per=0.47%, avg=4137.62, stdev=16528.11 write: io=2,044MB, bw=20,976KB/s, iops=5,244, runt= 99763msec slat (usec): min=12, max=17,396, avg=184.15, stdev=257.56 clat (usec): min=2, max=73,810, avg=2212.76, stdev=1479.10 bw (KB/s) : min= 0, max=36272, per=1.13%, avg=3318.54, stdev=8172.56 cpu : usr=2.74%, sys=93.90%, ctx=8540, majf=0, minf=368 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1567850/523158, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 50=0.01%, 100=0.01%, 250=0.01% lat (usec): 500=0.09%, 750=0.50%, 1000=3.96% lat (msec): 2=54.35%, 4=33.40%, 10=7.21%, 20=0.43%, 50=0.04% lat (msec): 100=0.01%, 500=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2561 read : io=6,128MB, bw=58,758KB/s, iops=14,689, runt=106787msec slat (usec): min=11, max=23,133, avg=173.71, stdev=257.70 clat (usec): min=3, max=11,841K, avg=2470.54, stdev=65500.47 bw (KB/s) : min= 0, max=140056, per=0.59%, avg=5205.73, stdev=18953.32 write: io=2,040MB, bw=19,566KB/s, iops=4,891, runt=106787msec slat (usec): min=12, max=20,316, avg=175.69, stdev=262.82 clat (usec): min=4, max=11,841K, avg=2468.04, stdev=65543.80 bw (KB/s) : min= 0, max=47184, per=1.16%, avg=3408.06, stdev=8525.59 cpu : usr=2.60%, sys=83.36%, ctx=10825, majf=0, minf=358 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1568647/522361, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 50=0.01%, 100=0.01%, 250=0.01% lat (usec): 500=0.20%, 750=1.35%, 1000=5.61% lat (msec): 2=56.43%, 4=29.65%, 10=6.27%, 20=0.42%, 50=0.05% lat (msec): 100=0.01%, 250=0.01%, 500=0.01%, 1000=0.01%, >=2000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2566 read : io=6,125MB, bw=64,424KB/s, iops=16,105, runt= 97360msec slat (usec): min=11, max=28,794, avg=174.49, stdev=259.93 clat (usec): min=3, max=1,067K, avg=2188.75, stdev=8685.78 bw (KB/s) : min= 0, max=104240, per=0.50%, avg=4371.86, stdev=17166.21 write: io=2,043MB, bw=21,484KB/s, iops=5,371, runt= 97360msec slat (usec): min=12, max=25,628, avg=175.58, stdev=254.95 clat (usec): min=3, max=1,065K, avg=2159.64, stdev=6795.08 bw (KB/s) : min= 0, max=34528, per=1.09%, avg=3201.77, stdev=8151.24 cpu : usr=2.46%, sys=92.14%, ctx=6923, majf=0, minf=358 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1568076/522932, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01% lat (usec): 250=0.01%, 500=0.08%, 750=0.42%, 1000=3.97% lat (msec): 2=57.99%, 4=31.63%, 10=5.43%, 20=0.40%, 50=0.04% lat (msec): 100=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%, 2000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2568 read : io=6,127MB, bw=62,184KB/s, iops=15,545, runt=100892msec slat (usec): min=12, max=24,702, avg=184.77, stdev=262.08 clat (usec): min=3, max=835K, avg=2243.55, stdev=1722.47 bw (KB/s) : min= 0, max=99480, per=0.48%, avg=4246.35, stdev=16608.96 write: io=2,041MB, bw=20,717KB/s, iops=5,179, runt=100892msec slat (usec): min=12, max=24,701, avg=186.54, stdev=267.69 clat (usec): min=8, max=50,242, avg=2235.46, stdev=1442.06 bw (KB/s) : min= 0, max=33400, per=1.02%, avg=2994.72, stdev=7757.57 cpu : usr=2.68%, sys=94.22%, ctx=8180, majf=0, minf=368 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1568463/522545, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01% lat (usec): 250=0.01%, 500=0.10%, 750=0.46%, 1000=3.70% lat (msec): 2=53.45%, 4=34.21%, 10=7.63%, 20=0.40%, 50=0.04% lat (msec): 100=0.01%, 250=0.01%, 750=0.01%, 1000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2571 read : io=6,127MB, bw=58,672KB/s, iops=14,668, runt=106933msec slat (usec): min=11, max=26,233, avg=173.92, stdev=270.82 clat (usec): min=3, max=10,788K, avg=2433.15, stdev=56084.02 bw (KB/s) : min= 0, max=153856, per=0.52%, avg=4580.43, stdev=18033.52 write: io=2,041MB, bw=19,545KB/s, iops=4,886, runt=106933msec slat (usec): min=12, max=26,232, avg=176.49, stdev=282.94 clat (usec): min=2, max=10,788K, avg=2603.59, stdev=70235.11 bw (KB/s) : min= 0, max=50616, per=1.11%, avg=3251.19, stdev=8439.40 cpu : usr=2.41%, sys=83.42%, ctx=10540, majf=0, minf=366 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1568497/522511, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 50=0.01%, 100=0.01%, 250=0.02% lat (usec): 500=0.29%, 750=1.67%, 1000=5.98% lat (msec): 2=57.91%, 4=27.24%, 10=5.97%, 20=0.85%, 50=0.06% lat (msec): 100=0.01%, 250=0.01%, 1000=0.01%, 2000=0.01%, >=2000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2575 read : io=6,123MB, bw=59,671KB/s, iops=14,917, runt=105073msec slat (usec): min=12, max=22,307, avg=190.15, stdev=281.91 clat (usec): min=3, max=985K, avg=2325.21, stdev=4263.24 bw (KB/s) : min= 0, max=134784, per=0.51%, avg=4465.73, stdev=17179.67 write: io=2,045MB, bw=19,931KB/s, iops=4,982, runt=105073msec slat (usec): min=13, max=22,308, avg=192.36, stdev=286.43 clat (usec): min=5, max=984K, avg=2325.95, stdev=4453.78 bw (KB/s) : min= 0, max=45200, per=1.02%, avg=2986.76, stdev=7837.42 cpu : usr=2.78%, sys=93.13%, ctx=8545, majf=0, minf=359 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1567451/523557, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01% lat (usec): 250=0.02%, 500=0.14%, 750=0.95%, 1000=5.31% lat (msec): 2=53.83%, 4=29.84%, 10=8.86%, 20=0.98%, 50=0.06% lat (msec): 100=0.01%, 500=0.01%, 750=0.01%, 1000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2578 read : io=6,127MB, bw=60,637KB/s, iops=15,159, runt=103462msec slat (usec): min=11, max=24,010, avg=187.54, stdev=267.45 clat (usec): min=3, max=614K, avg=2306.54, stdev=3870.17 bw (KB/s) : min= 0, max=128480, per=0.43%, avg=3826.70, stdev=15713.78 write: io=2,041MB, bw=20,204KB/s, iops=5,051, runt=103462msec slat (usec): min=12, max=24,008, avg=189.02, stdev=265.42 clat (usec): min=4, max=613K, avg=2304.43, stdev=3913.98 bw (KB/s) : min= 0, max=43568, per=1.01%, avg=2958.06, stdev=7664.00 cpu : usr=2.79%, sys=93.40%, ctx=8121, majf=0, minf=365 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1568410/522598, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01% lat (usec): 250=0.02%, 500=0.12%, 750=0.90%, 1000=4.44% lat (msec): 2=50.07%, 4=36.48%, 10=7.33%, 20=0.56%, 50=0.05% lat (msec): 100=0.01%, 250=0.01%, 750=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2580 read : io=6,126MB, bw=55,126KB/s, iops=13,781, runt=113790msec slat (usec): min=12, max=21,659, avg=186.51, stdev=294.57 clat (usec): min=3, max=11,301K, avg=2622.98, stdev=63184.53 bw (KB/s) : min= 0, max=255064, per=0.54%, avg=4798.37, stdev=20558.38 write: io=2,042MB, bw=18,378KB/s, iops=4,594, runt=113790msec slat (usec): min=12, max=21,557, avg=188.72, stdev=286.56 clat (usec): min=22, max=11,301K, avg=2596.75, stdev=60542.75 bw (KB/s) : min= 0, max=83896, per=1.04%, avg=3066.39, stdev=9231.50 cpu : usr=2.64%, sys=84.00%, ctx=35733, majf=0, minf=356 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1568203/522805, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01% lat (usec): 250=0.02%, 500=7.40%, 750=6.99%, 1000=7.36% lat (msec): 2=38.21%, 4=25.60%, 10=13.67%, 20=0.68%, 50=0.05% lat (msec): 100=0.01%, 500=0.01%, 1000=0.01%, >=2000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2588 read : io=6,131MB, bw=67,903KB/s, iops=16,975, runt= 92456msec slat (usec): min=11, max=26,772, avg=168.33, stdev=229.38 clat (usec): min=2, max=977K, avg=2059.06, stdev=5459.86 bw (KB/s) : min= 0, max=111328, per=0.64%, avg=5634.86, stdev=19828.90 write: io=2,037MB, bw=22,562KB/s, iops=5,640, runt= 92456msec slat (usec): min=13, max=22,712, avg=169.62, stdev=222.60 clat (usec): min=3, max=977K, avg=2065.39, stdev=6119.42 bw (KB/s) : min= 0, max=36864, per=1.25%, avg=3676.28, stdev=8868.76 cpu : usr=2.82%, sys=93.79%, ctx=6430, majf=0, minf=349 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1569515/521493, short=0/0 lat (usec): 4=0.01%, 50=0.01%, 100=0.01%, 250=0.01%, 500=0.04% lat (usec): 750=0.45%, 1000=4.85% lat (msec): 2=61.24%, 4=27.98%, 10=5.07%, 20=0.32%, 50=0.03% lat (msec): 100=0.01%, 500=0.01%, 1000=0.01% randrw: (groupid=0, jobs=1): err= 0: pid=2599 read : io=6,123MB, bw=61,463KB/s, iops=15,365, runt=102016msec slat (usec): min=11, max=24,381, avg=184.89, stdev=262.77 clat (usec): min=3, max=973K, avg=2275.52, stdev=5303.92 bw (KB/s) : min= 0, max=114752, per=0.48%, avg=4224.03, stdev=16680.52 write: io=2,045MB, bw=20,524KB/s, iops=5,131, runt=102016msec slat (usec): min=12, max=24,380, avg=186.70, stdev=267.43 clat (usec): min=4, max=972K, avg=2279.80, stdev=5680.26 bw (KB/s) : min= 0, max=38352, per=1.16%, avg=3415.46, stdev=8256.67 cpu : usr=2.97%, sys=93.01%, ctx=8536, majf=0, minf=359 IO depths : 1=0.0%, 2=0.0%, 4=0.1%, 8=0.1%, 16=0.1%, 32=87.5%, >=64=12.5% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, 64=0.0%, >=64=0.0% issued r/w: total=1567555/523453, short=0/0 lat (usec): 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=0.01% lat (usec): 250=0.02%, 500=0.13%, 750=0.72%, 1000=4.79% lat (msec): 2=52.14%, 4=33.91%, 10=7.75%, 20=0.50%, 50=0.04% lat (msec): 100=0.01%, 1000=0.01% Run status group 0 (all jobs): READ: io=98,022MB, aggrb=861MB/s, minb=56,449KB/s, maxb=70,223KB/s, mint=91509msec, maxt=113790msec WRITE: io=32,666MB, aggrb=287MB/s, minb=18,818KB/s, maxb=23,371KB/s, mint=91509msec, maxt=113790msec Disk stats (read/write): sdb: ios=1570992/526078, merge=2/0, ticks=351692/144852, in_queue=496272, util=91.13% sdc: ios=1572517/524594, merge=3/0, ticks=510340/132744, in_queue=645383, util=94.00% sdf: ios=1573298/523843, merge=1/1, ticks=423040/120090, in_queue=543710, util=92.36% sdg: ios=1573447/523777, merge=1/1, ticks=517825/171550, in_queue=690113, util=94.20% sdj: ios=1572489/524739, merge=1/0, ticks=380283/131216, in_queue=513255, util=91.21% sdk: ios=1574152/523069, merge=0/0, ticks=415897/145661, in_queue=561750, util=92.40% sdn: ios=1573508/523738, merge=0/0, ticks=482623/146468, in_queue=629581, util=93.21% sdo: ios=1572543/524674, merge=0/0, ticks=518038/178062, in_queue=699814, util=93.85% sdr: ios=1573160/524087, merge=0/0, ticks=472194/156134, in_queue=628538, util=94.07% sds: ios=1572887/524358, merge=0/0, ticks=512695/160155, in_queue=673744, util=94.59% sdv: ios=1574006/523222, merge=0/0, ticks=412086/140019, in_queue=553846, util=91.52% sdw: ios=1574190/522957, merge=13/6, ticks=1028215/319336, in_queue=1338369, util=96.72% sdz: ios=1571843/525402, merge=0/0, ticks=531606/189539, in_queue=721936, util=94.44% sdaa: ios=1535462/511435, merge=1/0, ticks=396725/142481, in_queue=539675, util=90.01% sdad: ios=1573613/523652, merge=0/0, ticks=559885/197060, in_queue=758248, util=94.65% sdae: ios=1535093/511783, merge=0/0, ticks=396290/179216, in_queue=576360, util=91.36%