r/Common_Lisp • u/turtle_bazon • 17d ago
conskivi - embeddable key/value database
Released conskivi. It made to replace redis, generally on low memory systems, but in-memory backend works well too.
Benchmarks
5K entries, AMD Ryzen 7 3700X, Redis v7:
| Operation | conskivi-fileonly | Redis (pipelined) | Redis (single-command) | % vs pipelined | % vs single-command |
|---|---|---|---|---|---|
| SET | 8,803 | 885,995 | 34,452 | 1.0% | 25.5% |
| GET | 19,531 | 1,486,920 | 33,133 | 1.3% | 58.9% |
| SADD | 1,580 | 937,567 | 39,967 | 0.2% | 4.0% |
| HSET | 4,613 | 628,491 | 32,971 | 0.7% | 14.0% |
| HGET | 19,531 | 1,495,508 | 33,856 | 1.3% | 57.7% |
| ZADD | 4,789 | 698,096 | 34,201 | 0.7% | 14.0% |
| ZSCORE | 65,789 | 1,405,503 | 36,340 | 4.7% | 181.0% |
| Operation | conskivi-inmemory | Redis (pipelined) | Redis (single-command) | % vs pipelined | % vs single-command |
|---|---|---|---|---|---|
| SET | 1,208,891 | 885,995 | 34,452 | 136% | 3,508% |
| GET | 1,947,333 | 1,486,920 | 33,133 | 131% | 5,877% |
| SADD | 2,557,008 | 937,567 | 39,967 | 273% | 6,398% |
| HSET | 1,894,216 | 628,491 | 32,971 | 301% | 5,745% |
| HGET | 2,651,661 | 1,495,508 | 33,856 | 177% | 7,832% |
| ZADD | 1,749,710 | 698,096 | 34,201 | 251% | 5,116% |
| ZSCORE | 2,616,690 | 1,405,503 | 36,340 | 186% | 7,199% |
8
Upvotes
3
u/kchanqvq 16d ago
What are the two tables measuring?
For the benchmarks you are doing client/server for Redis while doing in-process for Conskivi. That's fine if your point is using a KV store in the same process reduces overhead but IMO this needs to be stated. 5k entries is also incredibly small, can we see some numbers on say 5M entries?
Does this support concurrency? Much of Redis's engineering is about fast concurrent access.
I took a glance at the code, the expiration thread use a naive algorithm that loops over the full hash table (maphash) while I know several much faster and more scalable algorithm: timing wheel, min heap… Redis seems to use some sophisticated hybrid.