Unleashing the Dragon: Can Dragonfly Really Outpace Redis by 25x?

It seems that there are always a few projects claiming to have built the "fastest key/value store in the world," sometimes using phrases that are even more outrageous, like the following projects:

• Redis: https://github.com/redis/redis
• KeyDB: https://github.com/snapchat/keydb
• Dragonfly: https://github.com/dragonflydb/dragonfly
• Skytable: https://github.com/skytable/skytable

However, based on benchmark test results from abroad (30 threads, 1 client per thread for GET/SET), it is clear that some projects have exaggerated their claims. Let's take a look at the actual performance comparison:

1. Redis: 112,100 / 99,892
2. KeyDB: 288,931 / 282,997
3. Dragonfly: 408,322 / 392,446
4. Skytable: 619,992 / 676,091

Performance Comparison

Redis

I'm willing to call Redis the "original key"/value store (after memcached), as it is the oldest and most widely used in-memory database. As a long-time follower of Redis, I do know that it is single-threaded (and uses io-threads starting from 6.0), so its throughput is definitely somewhat limited compared to the other multi-threaded stores listed above. But Redis's biggest advantage is: it is the most feature-complete and oldest of all the systems here.

KeyDB

KeyDB claims to be a "multi-threaded branch of Redis that is 5 times faster". I really like this idea because I used to run multiple Redis instances on the same node and proxy them like a "single-node cluster" in order to improve CPU utilization. A single KeyDB instance can replace the unnecessary proxying, so many people have switched from Redis to KeyDB.

Dragonfly

Dragonfly claims to be 25 times faster than Redis (many developers say they can't reproduce this claim), and the official website says that "Dragonfly may be the fastest in-memory storage in the universe". It also supports Redis/Memcache commands, but I find it interesting mainly because of its performance. In addition, everyone wants to know why it is faster, and its official website clearly outlines the underlying architecture.

Dragonfly vs Redis Features

Skytable

I came across Skytable while looking for projects written in Rust, and it also claims to be "very fast". Skytable's "experimental benchmark" claims it is about 10 times faster than Redis and 2-3 times faster than KeyDB. I haven't heard much about Skytable, and it doesn't seem to be widely used.

Conclusion

Redis doesn't need much introduction and is known for being stable for use in production systems. KeyDB seems "stable enough" and is already being used by companies like Snapchat. No benchmark tests comparing Dragonfly to Skytable have been found yet. Of course, the best thing about Redis, KeyDB, and Skytable is that they don't make any outrageous assumptions about the systems they run on.

The reason for saying this is mainly because Dragonfly expects to run on servers with the latest hardware and kernel. But in reality, it's not very likely because both software and hardware replacements involve a process. And if the other three projects start adopting the latest features, they might also become much faster than they are now. Finally, Dragonfly and Skytable are still in the early stages of development, so it may be unfair to compare their features to Redis and KeyDB, which have been around for a longer time. Additionally, except for Skytable, these projects are backed by companies.