zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Camille Fournier <cami...@apache.org>
Subject Re: Support for large number of keys?
Date Tue, 27 May 2014 15:14:31 GMT
Well, ZK is not designed to be a database. I wouldn't recommend most people
try to use it as a database. There are many good KV store databases out
there that are better suited to the operations one wants to do with a
database and the consistency models and tradeoffs for a database.
I think it would be interesting to see the work, but I would not personally
recommend using ZK as a large datastore in this way.

C

On Tue, May 27, 2014 at 10:56 AM, Jan Kotek <discus@kotek.net> wrote:

> Hi,
>
> Zookeeper in current form stores all keys in memory and snapshots them
> periodically. I think it limits size of data Zookeeper can store.
>
> I am investigating feasibility of patching ZK to support large number of
> keys. It
> would use off-heap storage engine with incremental snapshotting.
> I think this way Zookeeper could store around 100 million keys-value pairs
> without
> negative impact on performance.
>
> Technically it is feasible (I already done something similar for
> Hazelcast). My
> question is if someone would actually use this improvement. Current ZK is
> probably just fine for most uses, it only has problem when you put
> excessive
> amount of data inside.
>
> So my questions is: Do you use ZK as a database? And do you have problem
> with long
> crash recovery time?
>
>
> Thanks,
> Jan Kotek
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message