hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Outdated data can not be cleaned in time
Date Thu, 09 Jun 2011 03:32:07 GMT
On Tue, Jun 7, 2011 at 12:41 AM, Zhoushuaifeng <zhoushuaifeng@huawei.com> wrote:
> https://issues.apache.org/jira/browse/HBASE-3723
> This issue is fixed and Committed to TRUNK, but not integrated in to 0.90.2 and 0.90.3,
this will causing outdated data not be cleaned in time.

Let me commit to branch.  Its a small change.

> For more, compaction checker will send regions to the compact queue to do compact. But
the priority of these regions is too low if these regions have only a few storefiles. When
there is large through output, and the compact queue will aways have some regions with higher
priority. This may causing the major compact be delayed for a long time(even a few days),
 and outdated data cleaning will also be delayed.
> If so , I suggested that the compaction checker sending regions need major compact to
the compact queue with higher priority.

I'd think that a region with more storefiles should take priority over
regions with a few files, even if these files are due for a major
compaction.  I can understand that if there are a lot of deletes in a
store, a major compaction could make a big difference but do you think
this the usual case?

Maybe the compaction algorithm should consider age of compactions too?
 If a compaction has been hanging the queue a good while, its priority
gets bumped a level?


View raw message