hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "E. Sammer" <e...@lifeless.net>
Subject Re: why not zookeeper for the namenode
Date Mon, 22 Feb 2010 18:12:54 GMT
On 2/22/10 12:53 PM, Zlatin.Balevsky@barclayscapital.com wrote:
> My 2 cents: If the NN stores all state behind a javax.Cache fa├žade it will be possible
to use all kinds of products (commercial, open source, facades to ZK) for redundancy, load
balancing, etc.

This would be pretty interesting from a deployment / scale point of view 
in that many jcache providers support flushing to disk and the concept 
of distribution and near / far data. Now, all of that said, this also 
removes some certainty from the name node contract, namely:

If jcache was used we:
  - couldn't promise data would be in memory (both a plus and a minus).
  - couldn't promise data is on the same machine.
  - can't make guarantees about consistency (flushes, PITR features, etc.).

It may be too general an abstraction layer for this type of application. 
It's a good avenue to explore, just playing devil's advocate.

Eric Sammer

View raw message