Skip to content

An abundance of IOPS and Zero Jitter

Two weeks ago, I wrote about The Data Center in the Age of Abundance and claimed that IOPS are – among other things – a solved problem.

What does a solved problem look like? Here is a benchmark running 100k random writes of 4K per second, with zero Jitter, at 350µs end-to-end write latency across six switches.

Databases really like reliably timed writes like these.

Maximum queue depth would be 48, the system is not touching that.

and here is iostat on the iSCSI client running the test

100k random writes, 4k write size, inside a 2 TB linux file of random data, on a 15 TB filesystem with XFS, on an LVM2 volume provided by iSCSI over a single 10 GBit/s interface, with six switch hops between the linux client and the array.

The array claims 150µs latency, on the linux we measure around 350µs. Out of that, there are less than 50µs from the switches and 150µs or more from the Linux storage stack (and that is increasingly becoming an issue).
Tested product was a Purestore Flasharray-X, client was Dell PowerEdge R630, 2x E5-2620v4, 128G, 10GBit/s networking. Thanks, Peter Buschman!
Published inContainers and KubernetesData Centers

6 Comments

  1. Olli

    Habt ihr nur eine IOPS-Lizenz fuer 100k gekauft, oder wieso ist der Wert so exakt?

    • 100k war stabil zu erreichen mit writes und reads auf einem LUN. Daher ist 100k unser Wahl für einen langen Test mit milliarden von IOs.

      Max erreicht war 115k bei writes und 106k bei reads.

      –Peter

  2. Was ist das für ein Test-Driver? Iscsi user Mode Client?

    Habt ihr auch bemerkt dass mit den. 10GE Karten die polling/NIO/wasauchimmer machen Jumbo Frames keinen echten Unterschied machen?

    • vdbench ohne Jumbo Frames

      Jumbo Frames braucht kein Mensch. Port-to-port Latenz ist sowieso nur mit <=1500 MTU garantiert vom Switch her.

  3. kris kris

    Peter points me to this article, in which the author tries to create a 100P device and blows up Linux Filesystems. It’s from 2012, so things may be better by now.

Leave a Reply

Your email address will not be published. Required fields are marked *