hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nitay Joffe (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1147) Modify the scripts to use Zookeeper
Date Mon, 26 Jan 2009 18:53:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12667348#action_12667348

Nitay Joffe commented on HBASE-1147:

Please see my comments in 1144. I think our configuration scheme is too complicated.

IRC conversation with Stack about this:

Jan 26 10:20:21 <St^Ack>	So, whats this about:
Jan 26 10:20:22 <nitay>	which is really just ripping code out of ZK's repo
Jan 26 10:20:32 <St^Ack>	The list of quorum servers specified in <code>${HBASE_HOME}/conf/zoo.cfg</code>
should also be written to <code>${HBASE_HOME}/conf/zookeepers</code>.
Jan 26 10:20:39 <St^Ack>	"The list of quorum servers specified in <code>${HBASE_HOME}/conf/zoo.cfg</code>
should also be written to <code>${HBASE_HOME}/conf/zookeepers</code>.
Jan 26 10:20:43 <St^Ack>	.."
Jan 26 10:20:49 <St^Ack>	info in two places?
Jan 26 10:20:55 <nitay>	three actually
Jan 26 10:21:05 <nitay>	that's why i wrote in my comment we should fix that
Jan 26 10:21:07 *	idefine (i=root@twizzler.dreamhost.com) has joined #hbase
Jan 26 10:21:13 <nitay>	so zoo.cfg is purely for ZK
Jan 26 10:21:16 *	St^Ack HHHRNHH!!!! wrong answer
Jan 26 10:21:29 <nitay>	well its a part of 1147
Jan 26 10:21:39 <nitay>	i was commenting on it saying we should discuss it
Jan 26 10:21:49 <nitay>	zookeepers is for HBase, its analogous to regionservers file
Jan 26 10:22:06 *	St^Ack looking
Jan 26 10:22:07 <nitay>	and then there's the option in hbase-site.xml
Jan 26 10:22:12 <nitay>	read my whole comments
Jan 26 10:22:34 <nitay>	specifically, https://issues.apache.org/jira/browse/HBASE-1144?focusedCommentId=12667125#action_12667125
Jan 26 10:23:05 <St^Ack>	oh, you are on it already
Jan 26 10:23:11 <St^Ack>	sorry missed that
Jan 26 10:23:16 <idefine>	Hi all, quick question (new to hadoop/hbase). If I have a
bunch of data in a database, do I need to write that data to disk inorder to use hadoop to
map/reduce? or can the map function pull from the database by offsetting that starting index?
Jan 26 10:23:37 <idefine>	the starting index*
Jan 26 10:23:49 <nitay>	so most of that is part of 1147, i wrote the overview.html to
match the current way its done, but i think really we should just have zoo.cfg, and parse
everything from there
Jan 26 10:26:34 <St^Ack>	nitay: just commented.  you were on the ball.  i was looking
at the cheerleaders
Jan 26 10:27:28 <St^Ack>	idefine: I believe there db hookups if you go looking through
hadoop; if not in hadoop itself or in contrib, then look in JIRAs for a patch
Jan 26 10:27:38 <nitay>	St^Ack, by populated u mean the whole host:port goodness? cause
i was just thinking of a true/false flag
Jan 26 10:27:57 <nitay>	if true, parse clientPort= and server.X= out of zoo.cfg and
start/stop with Hbase
Jan 26 10:28:09 <nitay>	if not, still have to parse those, but dont try to start/stop
Jan 26 10:28:22 <St^Ack>	idefine: my faint knowledge comes of quick peruse of hadoop
mailing list subject lines so don't put to much weight on it
Jan 26 10:29:03 <St^Ack>	nitay: how do hbase clients know which  remote zk cluster?

Jan 26 10:29:34 <idefine>	St^Ack: what if I am using hbase for my database, would it
make a difference?
Jan 26 10:30:09 <St^Ack>	idefine: in that case, there are existant hookups.  See under
the hbase mapred package 
Jan 26 10:30:19 *	St^Ack goes to reread nitay comment
Jan 26 10:30:24 <idefine>	St^Ack: thanks, will do.
Jan 26 10:30:40 <nitay>	St^Ack, oh b/c hbase clients have just the HBaseConfiguration
thing in the code u're saying, no zoo.cfg? 
Jan 26 10:31:37 <St^Ack>	nitay: no, zoo.cfg will be on classpath if its in conf dir.
Jan 26 10:31:47 <nitay>	St^Ack, right ye so im confused
Jan 26 10:32:00 <nitay>	why u need to ever specify ZK host:port in hbase-X.xml?
Jan 26 10:32:01 <St^Ack>	nitay: so, if a remote cluster, zoo.cfg will contain the addresses
of the remote zk quorum?
Jan 26 10:32:06 <nitay>	yes
Jan 26 10:32:10 <St^Ack>	ok.
Jan 26 10:32:13 <nitay>	it has to, that's how zookeeper works
Jan 26 10:32:18 <St^Ack>	then boolean as you suggest is way to go.
Jan 26 10:32:25 <nitay>	zoo.cfg:
Jan 26 10:32:25 <nitay>	tickTime=2000
Jan 26 10:32:25 <nitay>	dataDir=/var/zookeeper
Jan 26 10:32:25 <nitay>	clientPort=2181
Jan 26 10:32:25 <nitay>	initLimit=5
Jan 26 10:32:25 <nitay>	syncLimit=2
Jan 26 10:32:25 <nitay>	server.1=zoo1:2888:3888
Jan 26 10:32:25 <nitay>	server.2=zoo2:2888:3888
Jan 26 10:32:25 <nitay>	server.3=zoo3:2888:3888
Jan 26 10:32:49 <nitay>	so if u dont have any server.X lines its a local instance
Jan 26 10:33:01 <St^Ack>	I didn't realize that clients would read zoo.conf to get to
remote quorum also.
Jan 26 10:33:34 <nitay>	right i didnt think about that at first too, but then i realized
ll the info we need is in ZK's zoo.conf, untouched by us, just in different forms
Jan 26 10:33:46 <nitay>	i also think its important we never add stuff to zoo.cfg, keep
it exactly the same as ZK's
Jan 26 10:34:23 <St^Ack>	agreed
Jan 26 10:34:28 <St^Ack>	updated my comment in issue

> Modify the scripts to use Zookeeper
> -----------------------------------
>                 Key: HBASE-1147
>                 URL: https://issues.apache.org/jira/browse/HBASE-1147
>             Project: Hadoop HBase
>          Issue Type: Sub-task
>            Reporter: Jean-Daniel Cryans
>            Assignee: Jean-Daniel Cryans
>         Attachments: hbase-1147.patch

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message