hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1296) Bulk upload can overwhelm a datanode filling its disk because HDFS deletes lag
Date Fri, 26 Jun 2009 19:25:47 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724666#action_12724666

Andrew Purtell commented on HBASE-1296:

What HDFS jiras track this issue?

> Bulk upload can overwhelm a datanode filling its disk because HDFS deletes lag
> ------------------------------------------------------------------------------
>                 Key: HBASE-1296
>                 URL: https://issues.apache.org/jira/browse/HBASE-1296
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
> From a private mail with Jérôme Thièvre:
> "Two times I got the same kind of problems described by schubert zhang in the thread
*HDFS unbalance issue. (HBase over HDFS). *The first time, I found one of my regionserver
 with a full disk, whereas the others were at 10% of their capacity. I saw a lot of move file
command at hadoop level from the full regionserver to others, but after check on the hdfs,
I can see that hadoop copies the file but fails to delete it.  As the regionserver disk was
full, it didn't work anymore, even after hadoop and hbase restart. I had to delete all the
> "The second time this problem occurs I stopped the row insertion process before the disk
was full. After hadoop and hbase restart, hadoop has deleted the files and the system was
> It seems that in some cases where the system is heavily loaded with continuous writes
and compactions, hadoop can't remove files."

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message