hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-7932) Do the necessary plumbing for the region locations in META table and send the info to the RegionServers
Date Tue, 07 May 2013 21:59:16 GMT

     [ https://issues.apache.org/jira/browse/HBASE-7932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Devaraj Das updated HBASE-7932:
-------------------------------

    Attachment: 7932-simple-with-meta-pb.1.txt

Ok this patch should address most of Stack's previous comments except for the unit tests one.
I have a unit test that does end-to-end verification of this whole thing but not at the level
of racks and so on. Since a significant part of the 'math' is based on the rack configuration,
etc., I'll try to write some unit tests that fake racks and so on but I wanted to get a review
on the patch as it stands now. 

On the specific questions:
bq. I was thinking that when balance is called, your new balancer should have nothing to do
right?

I have put in a comment in the balanceCluster cluster call in FavoredNodeLoadBalancer. The
work on the balancer core is going to be done in a follow up jira.

bq. Does access to rackToRegionServerMap need to be synchronized

I have changed the patch so that all these data structures are in a separate class (FavoredNodesAssignmentHelper).
An instance of this class is created per a call to the balancer methods (via the assignmanager's
methods), and this class is seeded with the servers by the balancer. My previous patch had
this bug where it would not distinguish between servers based on what the AssignmentManager
chooses to use for a particular assignment.

bq. return serverSize >= FAVORED_NODES_NUM;

Done.

bq. On assign, you are just trying to spread the regions. I suppose it would make some sense
to consider how loaded a regionserver already is before choosing it? How many regions it is
carrying?

The hosts are decided on by the AssignmentMananger and passed to the FavoredNodesAssignmentHelper
via the FavoredNodeLoadBalancer.. The AM would choose the right hosts and give it to us..
The FavoredNodeLoadBalancer.balanceCluster call would look at the attributes of the regions/nodes
but that is not part of this patch now.

bq. So what if it is cluster startup and there 10k regions to assign.

Yeah.. I reverted this change and send the hosts to the RegionServers as part of the openRegion
request.

bq. If .META. already has been marked with favored nodes, we do not seem to be taking notice
only overwriting whatever is there on each assign

Fixed that (when a cluster starts up it looks at the favored nodes in AssignmentManager.assignAllUserRegions)

bq. I do not see how it is added to updateRegionFavoredNodesMapping.... am I reading it wrong?

Removed this in the new patch since the nodes are sent in the RPC as opposed to meta scan.

bq. Why InetSocketAddress saving favored nodes

The new HDFS create API works with InetSocketAddress and hence I thought we could start with
that itself. (HBASE-7942 would start using the new HDFS API.)

bq. I think this approach much cleaner; better packaging of all that is going on.

Cool.
                
> Do the necessary plumbing for the region locations in META table and send the info to
the RegionServers
> -------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-7932
>                 URL: https://issues.apache.org/jira/browse/HBASE-7932
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Devaraj Das
>            Assignee: Devaraj Das
>            Priority: Critical
>             Fix For: 0.95.1
>
>         Attachments: 7932-1.patch, 7932-3.patch, 7932-8.1.patch, 7932-8.patch, 7932-simple-2.patch,
7932-simple-with-meta-pb.1.txt, 7932-wip-2.patch, 7932-wip-3.patch, 7932-wip-4.patch, 7932-wip.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message