hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhoushuaifeng <zhoushuaif...@huawei.com>
Subject Outdated data can not be cleaned in time
Date Tue, 07 Jun 2011 07:41:13 GMT

This issue is fixed and Committed to TRUNK, but not integrated in to 0.90.2 and 0.90.3, this
will causing outdated data not be cleaned in time.
If outdated data not cleaned, scanners may do a lot of useless work and frequently LRU eviction,
and occupy too much filesystem IO,  even will cause lease expired and other operation Exception(not
only scan , even put).
So, I think it's better to integrated this patch into the future versions.

For more, compaction checker will send regions to the compact queue to do compact. But the
priority of these regions is too low if these regions have only a few storefiles. When there
is large through output, and the compact queue will aways have some regions with higher priority.
This may causing the major compact be delayed for a long time(even a few days),  and outdated
data cleaning will also be delayed.
If so , I suggested that the compaction checker sending regions need major compact to the
compact queue with higher priority.

Zhou Shuaifeng(Frank)

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message