zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rakesh R <rake...@huawei.com>
Subject RE: zookeeper in hadoop
Date Mon, 18 Aug 2014 08:24:21 GMT
Hi,

AFAIK following are few cases where ZK is used in Hadoop & BookKeeper components.

Hadoop is using ZK to provides the high-availability of the Master process like,

1) HDFS is using ZK for making the NameNode highly available. 
Also, BKJM(uses BookKeeper as journal manager) which is a sub-module in the HDFS to store
WAL(editlog transactions)
http://hadoop.apache.org/docs/r2.3.0/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailabilityWithNFS.html

2) YARN is using ZK for making the ResourceManager highly available. 
Also, they are using it for storing the state of the RM in ZooKeeper(ZKRMStateStore: A ZooKeeper-based
state-store implementation)
http://hadoop.apache.org/docs/r2.4.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html#Configurations


BookKeeper is using ZK for :

1) service discovery - to know the details about the running BK servers
2) metadata storage - keeps all the meta data about the ledgers(user data)
3) Auto-recovery - handle missing replicas to improve the fault-tolerance.
http://zookeeper.apache.org/bookkeeper/


Regards,
Rakesh
-----Original Message-----
From: mufc_fan [mailto:rajeshkumarit8292@gmail.com] 
Sent: 18 August 2014 11:25
To: zookeeper-user@hadoop.apache.org
Subject: zookeeper in hadoop

I am new to zookeeper and from the documentation I learnt that "zookeeper is a sub-project
of hadoop". so zookeeper must be used in hadoop. Can any one share their knowledge how zookeeper
is used in hadoop with simple example.



--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/zookeeper-in-hadoop-tp7580179.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

Mime
View raw message