hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Replication state
Date Mon, 13 Jun 2011 14:21:22 GMT
> From: Jason Rutherglen <jason.rutherglen@gmail.com>
> Right thanks.  I think replication is fairly simple, I don't know much
> about the HDFS sync code, if one has sync'd on the HLog writer, then
> an HLog reader should be able to read from there?

See my comments on HBASE-2357 regarding the undesirability of tailing HLogs directly from
HDFS, IMHO. Goes something like:

"[Read] slaves would need to access foreign store files for regions that are not open on the
RS. So then tailing HLogs, more foreign files, at the slave is not unreasonable. But that
is a major violation of assumptions that store files are private. Sharing store files will
require a coordination dance between master and slaves upon compaction and flushes. Sharing
active HLogs is more evil given the [HMaster] may become involved"

i.e. splitting upon failure of the master RS. But the situation is more complicated, now we
have distributed splitting (HBASE-1364).


"Also, the trouble with watching the WAL either on the slave side at the file or on the master
side with WALObserver is that .writeToWAL(false) edits will be unnoticed until flush."

   - Andy

View raw message