hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liujia <liujiaatb...@yahoo.cn>
Subject The Isolation and allocation part of HBase is availabe.
Date Wed, 27 Jul 2011 10:55:43 GMT
Now The Isolation and allocation part of HBase is available.
The issue is discussed in https://issues.apache.org/jira/browse/HBASE-4120,the source code
and patched tarball are located in https://github.com/ICT-Ope/HBase_allocation, Two versions
Integrated with HBase 0.90.2 and 0.90.3 are available to use.
The HBase isolation and allocation tool is designed to help users manage cluster resource
among different applications and tables.When we have a large scale of HBase cluster with many
applications running on it, there will be lots of problems. In Taobao.Inc there is a cluster
for many departments to test their applications' performance, these applications are based
on HBase. With one cluster which 
has 12 servers, there will be only one application running exclusively on this server, many
other applications must wait until the previous test finished.After we add allocation management
function to the cluster, applications can share the cluster and run concurrently. Also if
the Test Engineers want to make 
sure that there is no interference, they can move other tables out of this group.
Different groups can have different region server configurations, which means that some groups
optimized for reading can have large block cache size, and 
others optimized for writing can have large memstore size. Tables and region servers can
be moved easily between groups. After changing the configuration, a group can be restarted
alone instead of restarting the 
whole cluster.
In groups we use table priority to allocate resource. When system is busy, this can make sure
high-priority tables not affected by lower-priority ones
Though we put group and priority of tables together, users can use them separately, there
is no inseparable link between these two functions.
Suggestions and advice are welcome, please send mails to me or post your opinions on JIRA.
Jia Liu
Key Laboratory of Network Science and Technology,
Institute of Computing Technology,
Chinese Academy of Sciences,
P.O.Box 2704, Beijing 100190, China.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message