Hinting vs Heuristics: Plumbing I/O Cache Hints Through the Linux Storage Stack

The Linux-storage and wider storage community are actively investigating ways to express and leverage the varying performance characterics of storage devices. A storage device may do a better job servicing the I/O stream if it can discern details deeper than just the currently requested block address range. The T10 committee is in the process of specifying a hinting scheme to classify the in-flight data in a SCSI request. Similarly, a filesystem can do a better job of allocation if it is given some explicit hints from the application about how a file will be used. EXT4 is investigating an O_HOT/O_COLD hint that applications could use to express a coarse quality of service for a given file.

At the same time, bcache has arrived as a stacking block device driver that uses heuristics to guide the decision of whether an I/O request should be cached in a high performance device or passed on to the next tier in the storage hierarchy. This presentation investigates an approach to plumbing hints through the filesystem to be consumed by a modified bcache block device. The tradeoffs between hinting and heuristics, as well as a proposed mechanism for specifying cache policy in userspace, are explored. The target audience is kernel filesystem/block developers and application developers that want to express caching or other policies to a storage configuration.

Topic Lead: Dan Williams (<email address hidden>)
Dan is a Linux-storage developer at Intel. He contributed support for offloading raid5/6 calculations, developed bios-raid support for md/mdadm, and currently maintains the libsas based isci driver. He has presented at the Ottawa Linux Symposium, the Linux Storage Summit, and authored an article for LWN.net.