hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "ZooKeeper/MountRemoteZookeeper" by ebortnik
Date Wed, 15 Jun 2011 19:27:13 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "ZooKeeper/MountRemoteZookeeper" page has been changed by ebortnik:

  The performance of read and write accesses to the home partition remains unaffected in most
cases (see the details below), whereas accesses
  to remote partitions might suffer from lower latency/throughput. 
- Unlike the design proposed in http://wiki.apache.org/hadoop/ZooKeeper/PartitionedZookeeper
that addresses the same problem, we do not view the solution as
- partitioning a single namespace. Instead, our design provides a way for incorporating a
part of one ZK namespace in 
+ Our design provides a way for incorporating a part of one ZK namespace in 
  another ZK namespace, similarly to ''mount'' in Linux. Each ZK cluster has its own leader,
and intra-cluster requests are handled
  exactly as currently in ZK. Inter-cluster requests are coordinated by the local leader.
We strive to minimize the inter-cluster communication 
  required to achieve the consistency guarantees.
  === Background ===
+ Namespace partitioning for throughput increase has been partially addressed in ZK before.

+ The observer architecture allows read-only access to part of the namespace. The synchronization
+ with the partition's quorum happens in the background. Since the remote partition is read-only,
+ updates to it can be scheduled at arbitrary timing, and the consistency is preserved. The
+ is restricted access semantics. Our solution can reuse the synchronization building block
+ for observers. 
+ The design in http://wiki.apache.org/hadoop/ZooKeeper/PartitionedZookeeper discusses how
to detect
+ the need for partition automatically. It does not deal with the question of reconciling
multiple partitions 
+ (containers). Hence, this discussion is largely orthogonal to our proposal. 
  Several Distributed Shared Memory (DSM) systems proposed in 1990's implemented sequential
consistency, but hit
  performance bottlenecks around false sharing. ZK is a different case, because the namespace
is structured as a filesystem, 

View raw message