bookkeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enrico Olivelli <eolive...@gmail.com>
Subject BookKeeper as low-latency distributed data storage
Date Wed, 03 Aug 2016 14:10:52 GMT
Hi BookKeepers,

Recently JV explained how BookKeeper can support a long term
low-latency data store.

I'm working on a new project which need a fast replicated "object
store" on local LAN, replication is needed only for high-availability.

Usually the object (byte[]) size will be between 10KB and 10MB and
typically they are kept only for a couple of days, for a medium total
storage size of 700GB.

My idea is to write these objects to BookKeeper and store the entry
coordinates (ledgerId/offset) as a pointer to the object.

My question is about random access reading from Bookies, can I hope to
achieve low latency while reading a single entry from a ledger ?

Both writes and reads will run concurrently with an high degree of
parallelism (thousands of concurrent writes/reads)

In my use case it is very likely that reads will be done on recently
written data, so I think that the bookie cache would come to help

What do you think ?

Mime
View raw message