ORACLE UEK 4 : Where is my I/O scheduler ? None ? [Multi-queue model blk-mq]

If you upgraded to the new Unbreakable Enterprise Kernel Release 4,then if you take a quick look at the I/O scheduler used by your block devices this is what you will  see :

cat /sys/block/sdb/queue/scheduler

none !! what ?

At first this may seem strange but after a few check this is indeed the result of the implementation of the new I/O block layer Multi-queue model. As stated in the  Release Notes of UEK 4 :

“The new, scalable multiqueue block layer subsystem (blk-mq) for supporting high performance SSD storage implements per-CPU submission queues for receiving I/O requests, which are directed to hardware submission queues. The separate per-CPU submission and hardware submission queues balances the I/O workload across multiple CPU cores and reduces latency. The design supports the interface and features of the traditional block layer, but it is also capable of supporting many millions of I/O operations per second by taking advantage of the capabilities of NVM-Express or high-end PCI-E devices and multicore CPUs.”

This model is not quite new as it was introduced with Linux Kernel 3.13. But why a new model ? And what is this multi-queue thing ?

Basically the old single queue model has become a bottleneck when used with modern devices, capable of handling possibly millions of IOPs, due to many reason like High lock contention and Bad NUMA scaling.I will not recopy all the stuff only redirect you to this  very well explained document doc1 , doc2 , doc3

Capture 1

Capture 2

The new multi-queue model is the default in UEK 4 and it’s enabled for all the block devices (You can not have one using the old model and the other using the new one).In fact, the Linux kernel configuration parameter CONFIG_SCSI_MQ_DEFAULT is set to Y.

Capture 0

“This option enables the new blk-mq based I/O path for SCSI devices by default. With the option the scsi_mod.use_blk_mq module/boot option defaults to Y, without it to N, but it can still be overridden either way.” ref

So if we want to switch back to legacy I/O block layer we can set scsi_mod.use_blk_mq to N in the boot option.

But where is my scheduler ? Simply multi-queue model does not support I/O scheduling yet but may be soon (ex : mq-deadline) . So you may see some performance degradation with  rotational devices (slow devices) where I/O scheduling still have huge benefit.(like  turning random into sequential access to optimize performance on traditional hard drives)

“While  the  legacy  block  layer  takes  advantage  of  time  an IO  request  remains  in  the  submission  queue  to  sequentially reorder  requests,  the blk-mq design  simply  interleaves  IO requests  from  the  per-core  software  submission  queues  onto the hardware dispatch queues. With solid state devices, capable of higher random IO performance, the benefit from sequential ordering  is  lessened  and  the  FIFO  queues  avoid  locking overheads that become more pronounced at high IOP/s rates. This  trade-off  may  be  troublesome  for  rotational  media,  buas  noted  in  [3],  there  is  room  for  IO  scheduling  to  be  done between the two queue levels. To date, this scheduling has not been implemented. ” ref

I think it’s time to SLOB ! Have fun !

That’s it 😀

One thought on “ORACLE UEK 4 : Where is my I/O scheduler ? None ? [Multi-queue model blk-mq]

  1. Hello. how are you, I’m not an engineer I would know I have to say that parameters aggiungerea this usb ssd
    ID-2: USB / dev / sda model: Portable_SSD_T1 size: 250.1GB
    Thanks for your help
    i use archlinux

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s