hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: commit semantics
Date Tue, 12 Jan 2010 04:48:08 GMT


On Mon, Jan 11, 2010 at 8:12 PM, Joydeep Sarma <jssarma@apache.org> wrote:
> ok - hadn't thought about it that way - but yeah with a default of 1 -
> the semantics seem correct.
> under high load - some batching would automatically happen at this
> setting (or so one would think - not sure if hdfs appends are blocked
> on pending syncs (in which case the batching wouldn't quite happen i
> think) - cc'ing Dhruba).

Yes this is our version of group commit.

> if the performance with setting of 1 doesn't work out - we may need an
> option to delay acks until actual syncs .. (most likely we would be
> able to compromise on latency to get higher throughput - but wouldn't
> be willing to compromise on data integrity)

Good idea, we don't currently support that feature although we have
the opposite running by default which is deferred log flush. Tables
are never sync'ed and they rely on the LogSyncer thread awaitNanos'
timeout (configurable) or tables that are highly durable. In our
opinion, a cluster with a healthy mix of deferred and non-deferred
tables still guarantees a very high level of durability for the
default setting.

View raw message