virt: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=4
...
virt: (g=0): rw=randrw, bs=64K-64K/64K-64K/64K-64K, ioengine=libaio, iodepth=4
fio-2.1.2
Starting 4 processes
virt: Laying out IO file(s) (1 file(s) / 1024MB)
virt: Laying out IO file(s) (1 file(s) / 1024MB)
virt: Laying out IO file(s) (1 file(s) / 1024MB)
virt: Laying out IO file(s) (1 file(s) / 1024MB)

virt: (groupid=0, jobs=1): err= 0: pid=3714: Thu May 29 10:50:17 2014
  read : io=523520KB, bw=6395.4KB/s, iops=99, runt= 81860msec
    slat (usec): min=16, max=732, avg=36.84, stdev=33.84
    clat (msec): min=1, max=384, avg=11.48, stdev= 9.29
     lat (msec): min=1, max=384, avg=11.51, stdev= 9.29
    clat percentiles (msec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    6], 20.00th=[    6],
     | 30.00th=[    7], 40.00th=[    8], 50.00th=[   10], 60.00th=[   12],
     | 70.00th=[   14], 80.00th=[   17], 90.00th=[   21], 95.00th=[   24],
     | 99.00th=[   30], 99.50th=[   32], 99.90th=[   36], 99.95th=[  118],
     | 99.99th=[  383]
    bw (KB  /s): min= 1252, max=11324, per=25.04%, avg=6406.43, stdev=1538.94
  write: io=525056KB, bw=6414.8KB/s, iops=100, runt= 81860msec
    slat (usec): min=18, max=660, avg=46.04, stdev=38.11
    clat (msec): min=9, max=402, avg=28.37, stdev=11.62
     lat (msec): min=9, max=402, avg=28.41, stdev=11.62
    clat percentiles (msec):
     |  1.00th=[   14],  5.00th=[   16], 10.00th=[   19], 20.00th=[   21],
     | 30.00th=[   23], 40.00th=[   25], 50.00th=[   27], 60.00th=[   30],
     | 70.00th=[   32], 80.00th=[   36], 90.00th=[   41], 95.00th=[   45],
     | 99.00th=[   55], 99.50th=[   58], 99.90th=[  151], 99.95th=[  212],
     | 99.99th=[  404]
    bw (KB  /s): min= 1502, max= 7700, per=25.04%, avg=6424.42, stdev=999.39
    lat (msec) : 2=0.07%, 4=0.98%, 10=24.84%, 20=28.70%, 50=44.29%
    lat (msec) : 100=1.03%, 250=0.06%, 500=0.04%
  cpu          : usr=0.39%, sys=1.20%, ctx=14062, majf=0, minf=21
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=8180/w=8204/d=0, short=r=0/w=0/d=0
virt: (groupid=0, jobs=1): err= 0: pid=3715: Thu May 29 10:50:17 2014
  read : io=523520KB, bw=6403.1KB/s, iops=100, runt= 81750msec
    slat (usec): min=17, max=793, avg=37.14, stdev=30.71
    clat (msec): min=1, max=363, avg=11.40, stdev= 9.08
     lat (msec): min=1, max=363, avg=11.44, stdev= 9.08
    clat percentiles (msec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    6], 20.00th=[    6],
     | 30.00th=[    7], 40.00th=[    8], 50.00th=[   10], 60.00th=[   12],
     | 70.00th=[   14], 80.00th=[   17], 90.00th=[   21], 95.00th=[   24],
     | 99.00th=[   31], 99.50th=[   33], 99.90th=[   63], 99.95th=[  126],
     | 99.99th=[  363]
    bw (KB  /s): min=  627, max=11451, per=25.07%, avg=6414.10, stdev=1624.03
  write: io=525056KB, bw=6422.8KB/s, iops=100, runt= 81750msec
    slat (usec): min=21, max=798, avg=47.90, stdev=40.35
    clat (msec): min=8, max=344, avg=28.39, stdev=11.52
     lat (msec): min=9, max=344, avg=28.44, stdev=11.51
    clat percentiles (msec):
     |  1.00th=[   14],  5.00th=[   16], 10.00th=[   19], 20.00th=[   21],
     | 30.00th=[   23], 40.00th=[   25], 50.00th=[   27], 60.00th=[   30],
     | 70.00th=[   32], 80.00th=[   36], 90.00th=[   41], 95.00th=[   46],
     | 99.00th=[   55], 99.50th=[   58], 99.90th=[   67], 99.95th=[  243],
     | 99.99th=[  347]
    bw (KB  /s): min=  627, max= 7680, per=25.09%, avg=6437.73, stdev=1019.03
    lat (msec) : 2=0.01%, 4=0.93%, 10=25.46%, 20=28.27%, 50=44.19%
    lat (msec) : 100=1.04%, 250=0.05%, 500=0.04%
  cpu          : usr=0.41%, sys=1.19%, ctx=14016, majf=0, minf=21
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=8180/w=8204/d=0, short=r=0/w=0/d=0
virt: (groupid=0, jobs=1): err= 0: pid=3716: Thu May 29 10:50:17 2014
  read : io=523520KB, bw=6403.2KB/s, iops=100, runt= 81760msec
    slat (usec): min=16, max=707, avg=36.22, stdev=32.31
    clat (msec): min=2, max=370, avg=11.32, stdev= 9.67
     lat (msec): min=2, max=370, avg=11.36, stdev= 9.67
    clat percentiles (msec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    6], 20.00th=[    6],
     | 30.00th=[    7], 40.00th=[    8], 50.00th=[   10], 60.00th=[   12],
     | 70.00th=[   14], 80.00th=[   16], 90.00th=[   21], 95.00th=[   24],
     | 99.00th=[   30], 99.50th=[   32], 99.90th=[   40], 99.95th=[  198],
     | 99.99th=[  371]
    bw (KB  /s): min= 1001, max=11451, per=25.08%, avg=6416.32, stdev=1588.19
  write: io=525056KB, bw=6421.1KB/s, iops=100, runt= 81760msec
    slat (usec): min=21, max=644, avg=45.39, stdev=35.91
    clat (msec): min=10, max=360, avg=28.47, stdev=11.32
     lat (msec): min=10, max=360, avg=28.52, stdev=11.32
    clat percentiles (msec):
     |  1.00th=[   14],  5.00th=[   16], 10.00th=[   19], 20.00th=[   21],
     | 30.00th=[   23], 40.00th=[   25], 50.00th=[   28], 60.00th=[   30],
     | 70.00th=[   32], 80.00th=[   36], 90.00th=[   41], 95.00th=[   46],
     | 99.00th=[   55], 99.50th=[   58], 99.90th=[  129], 99.95th=[  169],
     | 99.99th=[  363]
    bw (KB  /s): min= 1252, max= 7680, per=25.08%, avg=6435.70, stdev=992.85
    lat (msec) : 4=1.04%, 10=25.74%, 20=27.95%, 50=44.18%, 100=0.98%
    lat (msec) : 250=0.05%, 500=0.05%
  cpu          : usr=0.40%, sys=1.18%, ctx=14084, majf=0, minf=21
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=8180/w=8204/d=0, short=r=0/w=0/d=0
virt: (groupid=0, jobs=1): err= 0: pid=3717: Thu May 29 10:50:17 2014
  read : io=523520KB, bw=6406.1KB/s, iops=100, runt= 81712msec
    slat (usec): min=15, max=681, avg=36.08, stdev=30.81
    clat (msec): min=2, max=417, avg=11.35, stdev= 9.06
     lat (msec): min=2, max=417, avg=11.38, stdev= 9.06
    clat percentiles (msec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    6], 20.00th=[    6],
     | 30.00th=[    7], 40.00th=[    8], 50.00th=[   10], 60.00th=[   12],
     | 70.00th=[   14], 80.00th=[   16], 90.00th=[   21], 95.00th=[   23],
     | 99.00th=[   30], 99.50th=[   32], 99.90th=[   52], 99.95th=[  129],
     | 99.99th=[  416]
    bw (KB  /s): min=  750, max=11451, per=25.10%, avg=6421.45, stdev=1626.62
  write: io=525056KB, bw=6425.8KB/s, iops=100, runt= 81712msec
    slat (usec): min=18, max=765, avg=45.84, stdev=38.96
    clat (msec): min=10, max=421, avg=28.42, stdev=11.80
     lat (msec): min=10, max=421, avg=28.47, stdev=11.80
    clat percentiles (msec):
     |  1.00th=[   14],  5.00th=[   16], 10.00th=[   19], 20.00th=[   21],
     | 30.00th=[   23], 40.00th=[   25], 50.00th=[   28], 60.00th=[   30],
     | 70.00th=[   32], 80.00th=[   36], 90.00th=[   41], 95.00th=[   46],
     | 99.00th=[   55], 99.50th=[   57], 99.90th=[  126], 99.95th=[  260],
     | 99.99th=[  420]
    bw (KB  /s): min= 1500, max= 7603, per=25.09%, avg=6436.60, stdev=1010.60
    lat (msec) : 4=1.01%, 10=25.35%, 20=28.49%, 50=44.05%, 100=0.99%
    lat (msec) : 250=0.05%, 500=0.05%
  cpu          : usr=0.38%, sys=1.21%, ctx=14033, majf=0, minf=20
  IO depths    : 1=0.1%, 2=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=8180/w=8204/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=2045.0MB, aggrb=25581KB/s, minb=6395KB/s, maxb=6406KB/s, mint=81712msec, maxt=81860msec
  WRITE: io=2051.0MB, aggrb=25656KB/s, minb=6414KB/s, maxb=6425KB/s, mint=81712msec, maxt=81860msec