hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From st...@duboce.net
Subject Re: Review Request: hbase-3019 Make bulk assignment on cluster startup run faster
Date Thu, 30 Sep 2010 23:11:30 GMT


> On 2010-09-30 14:38:49, Jonathan Gray wrote:
> > trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java, line
486
> > <http://review.cloudera.org/r/926/diff/1/?file=13151#file13151line486>
> >
> >     Have you seen this?  I guess this is kind of stuff that would happen on failed-over
master?

No. Haven't seen this.  Its a warn message.  If we see 'em something is up.


> On 2010-09-30 14:38:49, Jonathan Gray wrote:
> > trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java, line
604
> > <http://review.cloudera.org/r/926/diff/1/?file=13151#file13151line604>
> >
> >     So basic assumption here for startup is that master and no RS will fail.  I'm
fine with that.
> >     
> >     This bulk assign only done on startup, right?

Yes.  Has to be up for this bulk assign.  I think its fine if it crashes thereafter... we'll
soldier on.  To be proved in a test.


> On 2010-09-30 14:38:49, Jonathan Gray wrote:
> > trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java, line
612
> > <http://review.cloudera.org/r/926/diff/1/?file=13151#file13151line612>
> >
> >     No race here?  What if region server transitions to OPENING already before we
get here?

Chatting on irc, i like your suggestion of moving this state setting to before the rpc call.


> On 2010-09-30 14:38:49, Jonathan Gray wrote:
> > trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java, line
877
> > <http://review.cloudera.org/r/926/diff/1/?file=13151#file13151line877>
> >
> >     not sure i follow what it means if we time out, but should only happen if something
dies?  or takes a really long time?

Yes.  If something dies, the timeout will expire and then the timeout monitor will start looking
at regions in transition again and it'll timeout stuff that is failing to open... and assign
it elsewhere.  Thats the thought.


- stack


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/926/#review1368
-----------------------------------------------------------


On 2010-09-30 12:55:33, stack wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.cloudera.org/r/926/
> -----------------------------------------------------------
> 
> (Updated 2010-09-30 12:55:33)
> 
> 
> Review request for hbase.
> 
> 
> Summary
> -------
> 
> hbase-3019 Make bulk assignment on cluster startup run faster
> 
> M src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java
>   Disable test rather than comment out @Test.  junit complains no
>   test if do latter.
> M src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> M src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
>   openRegions: Added
> M src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
>   sendOpenRegions: Added
> M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
>   Add bulk assign method  Add ExecutorService to run bulk assignements.
> M src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java
>   Enhance logging.
> 
> 
> This addresses bug hbase-3019.
>     http://issues.apache.org/jira/browse/hbase-3019
> 
> 
> Diffs
> -----
> 
>   trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 1003192 
>   trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 1003192 
>   trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1003192 
>   trunk/src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java
1003192 
>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1003192

>   trunk/src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 1003192 
> 
> Diff: http://review.cloudera.org/r/926/diff
> 
> 
> Testing
> -------
> 
> I've been running patch up on testing cluster... starting and stopping then confirming
all regions on line by running rowcounter.
> 
> 
> Thanks,
> 
> stack
> 
>


Mime
View raw message