bookkeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sijie Guo <guosi...@gmail.com>
Subject Re: latency of bookkeeper
Date Fri, 06 Jul 2018 08:11:09 GMT
I think your question is a bit not clear, latency and throughput are two
kind of different metrics. Your question seems to be asking for high
throughput.

Anyway, I will try to explain the performance tradeoff between latency and
throughput and hope that helps.

Bookkeeper by default fsync the data to disks. It does 1ms group commit by
default to keep a good tradeoff between throughput and latency.

1) if you are using synchronous adds, since you are blocking on waiting
write response, your single thread throughput will be limited by the group
commit interval.
You can use multiple threads to improve throughput, since multiple writes
will be grouped together writing to disks, you will get as lower latency as
your group commit interval for your writes.

2) if your application can leverage asynchronous adds, you should consider
using asynchronous apis. It will offer you the best latency while be able
to achieve high throughput.

3) the latency will eventually be dominated by your disk fsync latency. SSD
or HDD with battery will have good fsync latency (about half millisecond).
However if your disk is not as good to
provide such lower latency, you can consider disable fsync and rely on
replication to achieve durability.
https://github.com/apache/bookkeeper/blob/master/conf/bk_server.conf#L309

Hope this helps.

On Thu, Jul 5, 2018 at 11:46 PM li.penghui@zhaopin.com.cn <
li.penghui@zhaopin.com.cn> wrote:

>
>     Hi
>
>             I try to use bookkeeper.  i care latency of write. so start a
> test in single thread.  get 400 ops/s  in double SSD.
>
>             how to improve performance to get the low-latency.
>
>     Thanks.
>
> ------------------------------
> li.penghui@zhaopin.com.cn
>

Mime
View raw message