hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rutherglen <jason.rutherg...@gmail.com>
Subject Re: Replication state
Date Mon, 13 Jun 2011 16:31:26 GMT
> Sharing store files will require a coordination dance between master and slaves upon compaction
and flushes. Sharing active HLogs is more evil given the [HMaster] may become involved

The log rollover happens relatively infrequently, however yes, when a
log rolls over, it could be tricky on the slave side.  We're assuming
that the slave RS has the same regions as the master RS.

With the Coprocessor system, each RS writes it's own HLogs, or perhaps
there's no need for the slave to write logs?

On Mon, Jun 13, 2011 at 7:21 AM, Andrew Purtell <apurtell@apache.org> wrote:
>> From: Jason Rutherglen <jason.rutherglen@gmail.com>
>> Right thanks.  I think replication is fairly simple, I don't know much
>> about the HDFS sync code, if one has sync'd on the HLog writer, then
>> an HLog reader should be able to read from there?
>
> See my comments on HBASE-2357 regarding the undesirability of tailing HLogs directly
from HDFS, IMHO. Goes something like:
>
> "[Read] slaves would need to access foreign store files for regions that are not open
on the RS. So then tailing HLogs, more foreign files, at the slave is not unreasonable. But
that is a major violation of assumptions that store files are private. Sharing store files
will require a coordination dance between master and slaves upon compaction and flushes. Sharing
active HLogs is more evil given the [HMaster] may become involved"
>
> i.e. splitting upon failure of the master RS. But the situation is more complicated,
now we have distributed splitting (HBASE-1364).
>
> and
>
> "Also, the trouble with watching the WAL either on the slave side at the file or on the
master side with WALObserver is that .writeToWAL(false) edits will be unnoticed until flush."
>
>   - Andy
>
>

Mime
View raw message