hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Kellerman (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-55) [hbase] Improve Master region assignment function
Date Wed, 13 Feb 2008 02:37:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12568412#action_12568412
] 

Jim Kellerman commented on HBASE-55:
------------------------------------

+1 on defining what is server load.

When we first defined it it was very simplistic and about the best we could do at the time.

Now we have multiple factors:
- update rate
- read rate
- memory pressure on region server

part of this will come from HBASE-70, but it is a more subtle problem, because you have to
take into
account hot spots (lots of updates), cold spots (almost no updates),  memory consumption on
the
region server, etc. 

Aside from #regions  being served by a region server and the number of requests it had served
since last heartbeat, none of these other considerations were in play.

What we need to do wrt region servers is come up with a metric that represents 'load factor'
If people have ideas on what that computation should be we'd be greatly interested in
hearing them.

Most of our test cases are pretty artificial, and do not represent a real world load, so trying
to come
up with a computation that tells us that a region server is too busy to accept more load is
difficult.


> [hbase] Improve Master region assignment function
> -------------------------------------------------
>
>                 Key: HBASE-55
>                 URL: https://issues.apache.org/jira/browse/HBASE-55
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Bryan Duxbury
>             Fix For: 0.2.0
>
>
> We would like the master's region assignment function to take into account more factors
when choosing where to assign regions.
>  
> - More advanced accounting of load on regionserver - memory, # requests, etc
> - Don't deploy both daughter regions to the same regionserver
> - Assign regions where the underlying DFS blocks are hosted if possible
> Please add additional ideas in comments as they come up.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message