accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: Read/Write Invariants Questions
Date Wed, 16 May 2012 21:10:16 GMT
On Wed, May 16, 2012 at 4:32 PM, Sukant Hajra <qn2b6c2b9w@snkmail.com> wrote:
> Excerpts from Keith Turner's message of Wed May 16 02:42:17 -0500 2012:
>>
>> After a call to flush() on a batchwriter returns, any mutations
>> written before the call to flush should be immediately visible.
>
> I don't want to belabor the point, but I just want to be sure I'm not
> interpreting your response too casually.  From your response, I'm now under the
> impression that a flush blocks until the server sends back an acknowledgment
> that the mutation has been written to the log.  Then all subsequent reads look
> not only at HDFS, but also the write logs to make sure they have the most
> consistent view?  Is this the case?  I appreciate the confirmation to save me a
> dig into the source code.

Reads look in the in memory map, not the walog, that Bill mentioned.
It sorted and supports efficient lookups.

>
> If the reads are truly immediately consistent, has there ever been talk of
> making inconsistent reads for the sake of improving read times?  Or is it all
> in the noise with respect to network speeds and not worth the effort?

Reads are consistent if you call flush beforehand.  If you are just
streaming a lot of mutations to a batchwriter and not flushing, you do
not know when the mutations will be visible.   This is a very
efficient way to write lots of data.  The frequency of flushing will
affect the write spead, but should not impact the read speed (other
than the fact that really frequent flushing may cause more disk
contention between write and read).

>
> Also, if flush blocks waiting for an acknowledgment, I'm assuming that the
> writer will throw a MutationsRejectedException.  If this happens, is the
> BatchWriter still usable?  Or should I close it out and get a new one?  The
> connector should be fine, though, right?  I'm just trying to make sure I have
> my error handling logic sanely configured.

I think it flush throws an exception that it leaves the batch writer
in state such that future calls to addMutation( ) will throw an
exception.

>
> Other than that, thanks a lot for your prompt responses.  They really helped.
>
> -Sukant

Mime
View raw message