How does this compare against Valkey?

[re-thc]

Hi, most of the benchmarks are against Redis. Valkey now supports parallel processing and other features.

How does Dragonfly compare?

[kostasrim]

Hi @re-thc

Valkey now supports parallel processing

From my understanding Valkey does not exactly support parallel processing. The commands you are running are still single threaded so once you saturate that you will hit a wall. From my current understanding parallel processing in Valkey, is not really parallel processing of commands but rather threads that deal with IO, command parsing etc. The idea is that the main thread executes commands while the rest of the IO threads parse, reply etc. That being said, this delegates a bunch of responsibility to the rest of the threads allowing the main thread to have time to execute more commands. However, this is not substitute for a shared memory (or a shared nothing) architecture where each of the threads can access any part of the redis dictionary (and actually execute the commands in parallel). That being said, there is still a hard limit on what you can actually parallize and what not as opposed to let's say dragonfly where each thread can execute commands.

There is also the memory side. Df does not use Redis Dictionary to store key/values but rather something called a DashTable which effectively has less metadata waste (meaning that an entry in the dash table requires less bytes than the redis dictionary). Furthermore, our shared nothing architecture allows parallel execution due and our transaction framework does not require complicate locking strategies alleviating a lot of the bottlenecks associated with the traditional lock managers found in shared memory systems.

All in all my two cents are -- and without any actual benchmarks-- I would expect DF would offer way better performance and memory characteristics.

[alexted]

Less words, more deeds. Will you do a benchmark comparison?

There are already independent benchmarks of Valkey on many third-party sites, and it shows excellent results, not inferior to yours. Considering that Valkey supports all Redis commands and, most importantly, can scale horizontally in a cluster deployment (supported out of the box), I wonder how dragonfly will look against it.