hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-1296) Bulk upload can overwhelm a datanode filling its disk because HDFS deletes lag
Date Mon, 30 Mar 2009 08:38:50 GMT
Bulk upload can overwhelm a datanode filling its disk because HDFS deletes lag
------------------------------------------------------------------------------

                 Key: HBASE-1296
                 URL: https://issues.apache.org/jira/browse/HBASE-1296
             Project: Hadoop HBase
          Issue Type: Bug
            Reporter: stack


>From a private mail with Jérôme Thièvre:

"Two times I got the same kind of problems described by schubert zhang in the thread *HDFS
unbalance issue. (HBase over HDFS). *The first time, I found one of my regionserver  with
a full disk, whereas the others were at 10% of their capacity. I saw a lot of move file command
at hadoop level from the full regionserver to others, but after check on the hdfs, I can see
that hadoop copies the file but fails to delete it.  As the regionserver disk was full, it
didn't work anymore, even after hadoop and hbase restart. I had to delete all the data.

"The second time this problem occurs I stopped the row insertion process before the disk was
full. After hadoop and hbase restart, hadoop has deleted the files and the system was operational.
It seems that in some cases where the system is heavily loaded with continuous writes and
compactions, hadoop can't remove files."



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message