hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liu Jia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4120) isolation and allocation
Date Fri, 29 Jul 2011 06:34:10 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072700#comment-13072700

Liu Jia commented on HBASE-4120:

@Jeff Thanks Jeff, I think to optimize the customer facing tables and control the resource
of batch processing tables it's a good use case for priority.
In Taobao Company, actually the group is used more often than table priority. Maybe without
test or competition on resource, the table priority works quiet. But the group has a user
portal and friendly with the users who not very familiar with HBase.
The department which holds a shared HBase cluster called Taobao data platform, and many other
departments may not very familiar with HBase want to use HBase. Most of them would like to
test the performance (mostly is the throughput per region server) with their data and methods
first. So the isolation part is very useful for them, 
I think if HBase wants to be a basic component of data center like Hadoop, a convenient and
flexible way to isolate different projects is important.
Jeff, is the mentioned part of the complicate of HBase significantly related with the Portal
and too many JSP pages?  What if add a shell tool to replace them? 
Because the actually implementation of group is just depends on region assignment and movement.

> isolation and allocation
> ------------------------
>                 Key: HBASE-4120
>                 URL: https://issues.apache.org/jira/browse/HBASE-4120
>             Project: HBase
>          Issue Type: New Feature
>          Components: master, regionserver
>    Affects Versions: 0.90.2
>            Reporter: Liu Jia
>         Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf,
HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, System Structure.jpg
> The HBase isolation and allocation tool is designed to help users manage cluster resource
among different application and tables.
> When we have a large scale of HBase cluster with many applications running on it, there
will be lots of problems. In Taobao there is a cluster for many departments to test their
applications performance, these applications are based on HBase. With one cluster which has
12 servers, there will be only one application running exclusively on this server, and many
other applications must wait until the previous test finished.
> After we add allocation manage function to the cluster, applications can share the cluster
and run concurrently. Also if the Test Engineer wants to make sure there is no interference,
he/she can move out other tables from this group.
> In groups we use table priority to allocate resource, when system is busy; we can make
sure high-priority tables are not affected lower-priority tables
> Different groups can have different region server configurations, some groups optimized
for reading can have large block cache size, and others optimized for writing can have large
memstore size. 
> Tables and region servers can be moved easily between groups; after changing the configuration,
a group can be restarted alone instead of restarting the whole cluster.
> git entry : https://github.com/ICT-Ope/HBase_allocation .
> We hope our work is helpful.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message