bookkeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sijie Guo <guosi...@gmail.com>
Subject Re: Expected workload for DBLedgerStorage ?
Date Wed, 20 Jun 2018 19:56:01 GMT
On Wed, Jun 20, 2018 at 12:45 PM Enrico Olivelli <eolivelli@gmail.com>
wrote:

>
>
> Il mer 20 giu 2018, 16:52 Sijie Guo <guosijie@gmail.com> ha scritto:
>
>> On Wed, Jun 20, 2018 at 4:09 AM Enrico Olivelli <eolivelli@gmail.com>
>> wrote:
>>
>>> Hi,
>>> I am experimenting DBLedgerStorage with some project.
>>> With my workloads I don't see much benefit, or sometimes a performance
>>> degradation,
>>>
>>
>> Do you have a performance comparison? Or can you describe what have you
>> observed?
>>
>
> No real 'sharable' numbers, just run a bunch of benchmarks of one
> application. My data has no real meaning.
>
>
>
>
>>
>>> just by keeping the same bookie and application configuration (just
>>> switching the storage manager classe name in the bookie configuration).
>>>
>>> Which is the expected workload for DBLedgerStorage ?
>>> (Usually I am using the default SortedLedgerStorage)
>>>
>>
>> Ideally it should be working out better when you have large number of
>> ledgers per bookie (let's say more than 10k ledgers).
>>
>
> My test created only a few ledger ( less then 10) so maybe this is why I
> see no benefit.
>
As far as I understand RocksDB is used for indexes and maybe with a few
> ledgers the overhead is greater than the gain in term of resource saving.
> I have to tune my test with more concurrency.
>

> In this project I will have concurrent readers and writers, but readers
> will access data at any point of ledgers, so no usual 'tailing reads'. I
> will be using ledgers more likes files than like streams.
>

Are you fetching data by entry ids or would you expect to fetch entries by
their *bytes* offset?

In other words, does this https://github.com/apache/bookkeeper/issues/1376
meet your requirement if you are looking for an interface to fetch entries
more about their "bytes" offset?


> So having rocks db to store offsets seems a very good chance for me.
> In real systems I will have thousands of active ledgers per bookie.
> I have to use a more complex suite of benchmarks for the use case.
>
> I will be back with more interesting comparisons
>
> Thank you
> Enrico
>
>
>
>
>>
>>>
>>> Cheers
>>> Enrico
>>>
>>>
>>>
>>> --
>
>
> -- Enrico Olivelli
>

Mime
View raw message