hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1062) Compactions at (re)start on a large table can overwhelm DFS
Date Sun, 14 Dec 2008 21:51:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656475#action_12656475
] 

stack commented on HBASE-1062:
------------------------------

+1 on it being critical.  Powerset has a cluster of ~5000 regions.  Start-up is a big-bang
but steady-state happens eventually.  I haven't done much study of it.  I can imagine that
indeed if cluster went down bad with a few major compactions in the mix, startup could be
messy.  How many regionservers Andrew?  And a HRS beside each datanode?  (Our nodes are relatively
lightly-loaded -- 50 or so regions in 2G heaps).

> Compactions at (re)start on a large table can overwhelm DFS
> -----------------------------------------------------------
>
>                 Key: HBASE-1062
>                 URL: https://issues.apache.org/jira/browse/HBASE-1062
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: regionserver
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> Given a large table, > 1000 regions for example, if a cluster restart is necessary,
the compactions undertaken by the regionservers when the master makes initial region assignments
can overwhelm DFS, leading to file errors and data loss. This condition is exacerbated if
write load was heavy before restart and so many regions want to split as soon as they are
opened. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message