zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Kelly <iv...@apache.org>
Subject Re: Running zookeeper with journaling disabled on an ext4
Date Wed, 21 Oct 2015 18:12:24 GMT
On Wed, Oct 21, 2015 at 8:00 PM, Edward Capriolo
<edward.capriolo@huffingtonpost.com> wrote:
> There used to be this setting in zk.
>
> zookeeper.forceSync
> Using Kafka I used to set this setting to false, The reason is I had a
> number of clients writing offsets to zk and the disk system was always
> sync-ing dirty pages. The system would be super high IO wait.
>
> I set this to false and disk usage basically dropped to 0.
forceSync means use fsync for zk journal operations. Without this,
writes to zookeeper are only going into the os page cache, so a crash
of the node, could cause data loss on that node. Even worse, if a node
crashes and reboots, it can accept two competing proposals and you can
theoretically end up with a system with splitbrain. This is why
durability is a requirement for all paxos like protocols.

-Ivan

Mime
View raw message