hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eason Lee <softse....@gmail.com>
Subject Re: Parent region data not deleted after compact
Date Thu, 05 Apr 2012 09:32:48 GMT
I have found this from maillist, maybe update to cdh3u3 or 0.90.5 will
solve this problem, thanks all

It's supposed to but there are a few leaks such as:



On Mon, Nov 21, 2011 at 7:32 PM, 吕鹏 <[EMAIL PROTECTED]> wrote:
> I set "hbase.hregion.max.filesize" to 9223372036854775807. When an average
> region size on HDFS reaches 6G, i split all the regions in my hbase table
> by using hbase java api HBaseAdmin.split. Before i do split, there are
> regions in my table, and when the split is finished, there a 2000
region is
> available as expected. But on the HDFS, there are 3000 region directories
> in the /hbase/tablename/columnfamily/*. Can hbase delete the old regions
> after region split automatically?
> *b2fb55650a67b28b791be4d6f813de8a *is one of the original regions before
> split, and the hbase master log is shown as follows:
> [hadoop@master3-hbase logs]$ grep b2fb55650a67b28b791be4d6f813de8a
> hbase-hadoop-master-master3-hbase.sii.cn.log
> 11/11/17 17:04:58 INFO master.ServerManager: Received REGION_SPLIT:
> pageDB,4a7ef985,1320941062356.b2fb55650a67b28b791be4d6f813de8a.:
> Daughters;
> from slave5-hbase.sii.cn,60020,1320980409810
> 11/11/17 23:45:30 INFO catalog.MetaEditor: Deleted daughter reference
> pageDB,4a7ef985,1321520697601.aa5828bfb105e1109aa676e5361b1793.,
> qualifier=splitA, from parent pageDB,4a7ef985,1320941062356.*
> b2fb55650a67b28b791be4d6f813de8a*.
> 11/11/18 00:10:30 INFO catalog.MetaEditor: Deleted daughter reference
> qualifier=splitB, from parent pageDB,4a7ef985,1320941062356.*
> b2fb55650a67b28b791be4d6f813de8a*.

> I have load mydata into hbase.
> After hbase calm down, I found that the data size is about 10 times the
> orignal data size.
> Finally I found the number of directory is /hbase/tableName is much more
> than the region number
> When I use hadoop fs -dus /hbase/tableName to see the size of the
> directorys, I found several dir take up more than 10GB spaces and they
> are not list in the table regions.In my opion, they are parent region
> directories that forget to clean.
> I'm try to do major_compact on this table now,but i don't know if this
> can solve the problem. Anyone can help to clear those dirs?

View raw message