zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Kotek <dis...@kotek.net>
Subject Support for large number of keys?
Date Tue, 27 May 2014 14:56:15 GMT

Zookeeper in current form stores all keys in memory and snapshots them 
periodically. I think it limits size of data Zookeeper can store.

I am investigating feasibility of patching ZK to support large number of keys. It 
would use off-heap storage engine with incremental snapshotting. 
I think this way Zookeeper could store around 100 million keys-value pairs without 
negative impact on performance.  

Technically it is feasible (I already done something similar for Hazelcast). My 
question is if someone would actually use this improvement. Current ZK is 
probably just fine for most uses, it only has problem when you put excessive 
amount of data inside. 

So my questions is: Do you use ZK as a database? And do you have problem with long 
crash recovery time?

Jan Kotek

  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message