hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Francis Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
Date Thu, 25 Oct 2012 00:34:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483768#comment-13483768

Francis Liu commented on HBASE-6721:

Thanks for the comments [~eclark]

I couldn't seem to get it to compile for me on 0.94
Did you apply the patches attached in the subtasks prior to apply this patch? If you'd like
them all in one single patch I can do that as well.

There seem to be a bunch of formatting changes that aren't needed.
Will clean that up in the next update.

Passing in the preferred server into the load balancer on randomAssignment is messy. If we
know the preferred server why call this function at all ?
Good point, will remove that argument.

The balancer is a public interface and we can't make changes to it in a minor release. And
this patch won't apply to trunk.
I see, we can make it binary compatible at least by supporting both interfaces if you're amenable
to that. We're planning on getting 0.94 into production and it'd be great if we didn't have
a lot of custom patches on top of it.

With this many interfaces and classes it might make sense to move them into a namespace.
Will look into doing this, my main concern is if there any dependencies to package private

Why a co-processor and not build it in ?
    Security was done that was because it can be added or removed. As the patch is that's
not really possible
    This makes a lot of changes in core code for something that is a co-processor.
As part of the design, HBase should run fine without the group based classes enabled (endpoint,
balancer, etc). If it is not that case then that's a bug. As for some code changes in core
code. Some may be unavoidable, but we could probably still make it less invasive (ie remove
the EventHandler changes). Having said that, I don't mind if the community would like to have
this fully integrated into HBase, just let us know.

Don't create a DefaultLoadBalancer in GroupBasedLoadBalancer. The balancer was made pluggable
and that feature shouldn't go away.
The balancer is still pluggable it's just not pluggable for the GroupBasedLoadBalancer. Though
should be ok to make that pluggable as well.

HTableDescriptor seems like the correct location for info about the table if you don't want
to put that data into meta.
Yes, we have group affiliation store as a table property. Though group information is stored
on hdfs.

putting things into the filesystem seems like the wrong way to do it. There are just so many
different moving parts with getting things from hdfs with caching and cache invalidation,
and edge cases on failure.
I see, were do you suggest we put it? Zookeeper? We mainly had it in HDFS since ZK, seemed
to be the place to store only ephemeral data? Putting the data in tables would be a lot more
complex and would require more core code change.

> RegionServer Group based Assignment
> -----------------------------------
>                 Key: HBASE-6721
>                 URL: https://issues.apache.org/jira/browse/HBASE-6721
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Francis Liu
>            Assignee: Vandana Ayyalasomayajula
>             Fix For: 0.96.0
>         Attachments: HBASE-6721_94.patch, HBASE-6721_94.patch, HBASE-6721-DesigDoc.pdf
> In multi-tenant deployments of HBase, it is likely that a RegionServer will be serving
out regions from a number of different tables owned by various client applications. Being
able to group a subset of running RegionServers and assign specific tables to it, provides
a client application a level of isolation and resource allocation.
> The proposal essentially is to have an AssignmentManager which is aware of RegionServer
groups and assigns tables to region servers based on groupings. Load balancing will occur
on a per group basis as well. 
> This is essentially a simplification of the approach taken in HBASE-4120. See attached

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message