hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7932) Do the necessary plumbing for the region locations in META table and send the info to the RegionServers
Date Tue, 07 May 2013 06:45:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13650564#comment-13650564

stack commented on HBASE-7932:

I was thinking that when balance is called, your new balancer should have nothing to do right?
 But I suppose it has to read .META. per region to balance and then assign to one of the favored
node locations (in a manner that helps w/ general cluster balancer?  Or teach the stochastic
balancer to pay attention to these .META. table additions?)

Does access to rackToRegionServerMap need to be synchronized since you are checking, then
calculating before putting back an item to two different maps (Or maybe access will be single
threaded when balancing?)?

Is this.regionServerToRackMap.keySet().size(); different from this.regionServerToRackMap.size()?

Instead of

+    if (serverSize < FAVORED_NODES_NUM)
+      return false;
+    return true;

maybe do....

return serverSize >= FAVORED_NODES_NUM;

What happens if one rack only?  i.e. 20 servers?  Hmmm... looks like things will work fine.
 We'll just spread the regions over the rack.

On assign, you are just trying to spread the regions.  I suppose it would make some sense
to consider how loaded a regionserver already is before choosing it?  How many regions it
is carrying?

So what if it is cluster startup and there 10k regions to assign.  Are we inlining this update
of .META. one region at a time as we process the 10k regions?  This could slow assign way
down adding at least a second per region?  I though you were doing similar on open of a region
doing read of favored nodes but it looks like this read-before-we-open has been in place w/
a while now, added by hbase-5094 in 0.92.  Doing this latter, piggybacking on the read that
is already happening, makes it so you do not have to complicate the open passing favored nodes
from master to regionserver?  (Thats good).

If .META. already has been marked with favored nodes, we do not seem to be taking notice only
overwriting whatever is there on each assign?  Do I have that wrong?

The first time you do this:

+            favoredServerList = getFavoredNodesList(favoredNodes);

I do not see how it is added to updateRegionFavoredNodesMapping.... am I reading it wrong?

favoredNodes in .META. need to be pb'd (then you can serialize them as ServerNames and not
have to do that parse of a String).

Why InetSocketAddress saving favored nodes?  I see you are doing unresolve which helps but
why use it at all?  Can you not use the hostname/port you got out of meta written by the master?

Would suggest figuring how to add unit tests for methods of FavoredNodeLoadBalancer since
they are doing a bunch of math?  Maybe make them static so easy to conjure inputs and test
the method outputs.

I think this approach much cleaner; better packaging of all that is going on.
> Do the necessary plumbing for the region locations in META table and send the info to
the RegionServers
> -------------------------------------------------------------------------------------------------------
>                 Key: HBASE-7932
>                 URL: https://issues.apache.org/jira/browse/HBASE-7932
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Devaraj Das
>            Assignee: Devaraj Das
>            Priority: Critical
>             Fix For: 0.95.1
>         Attachments: 7932-1.patch, 7932-3.patch, 7932-8.1.patch, 7932-8.patch, 7932-simple-2.patch,
7932-wip-2.patch, 7932-wip-3.patch, 7932-wip-4.patch, 7932-wip.patch

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message