bookkeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enrico Olivelli <>
Subject Re: BookKeeper as low-latency distributed data storage
Date Fri, 12 Aug 2016 12:09:02 GMT
Thanks Sijie. I will make some bench and  report the results


Il Mar 9 Ago 2016 21:50 Sijie Guo <> ha scritto:

> In general, reading an entry from bookie is pretty fast as most of the
> index pages will be cached in memory, so it typically only needs one disk
> access.
> You might see evaluated latency when index pages begin swapping in and
> out. So ideally I would suggest using small index page size and large
> memory, so you can keep most of the index pages in memory.
> - Sijie
> On Wed, Aug 3, 2016 at 7:10 AM, Enrico Olivelli <>
> wrote:
>> Hi BookKeepers,
>> Recently JV explained how BookKeeper can support a long term
>> low-latency data store.
>> I'm working on a new project which need a fast replicated "object
>> store" on local LAN, replication is needed only for high-availability.
>> Usually the object (byte[]) size will be between 10KB and 10MB and
>> typically they are kept only for a couple of days, for a medium total
>> storage size of 700GB.
>> My idea is to write these objects to BookKeeper and store the entry
>> coordinates (ledgerId/offset) as a pointer to the object.
>> My question is about random access reading from Bookies, can I hope to
>> achieve low latency while reading a single entry from a ledger ?
>> Both writes and reads will run concurrently with an high degree of
>> parallelism (thousands of concurrent writes/reads)
>> In my use case it is very likely that reads will be done on recently
>> written data, so I think that the bookie cache would come to help
>> What do you think ?
> --

-- Enrico Olivelli

View raw message