hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhoushuaifeng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3969) Outdated data can not be cleaned in time
Date Fri, 17 Jun 2011 03:06:48 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050879#comment-13050879
] 

zhoushuaifeng commented on HBASE-3969:
--------------------------------------

What will happen if we seting major priority between 1 and blockingStoreFiles - compactionThreshold
may be like this: for example, blockingStoreFiles = 7, compactionThreshold = 3, we set major
compact priority to 2:
If some region have 3 or 4 files, it's priority is lower than the major compact, the major
compact will do first. But if the files in this region increase to 6 or 7, it's priority will
be 1 or 0, the compact on this region will do first. So, the major compact will have a little
higher priority but will not causing other regions blocked. Is it satisfactory?

> Outdated data can not be cleaned in time
> ----------------------------------------
>
>                 Key: HBASE-3969
>                 URL: https://issues.apache.org/jira/browse/HBASE-3969
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.90.1, 0.90.2, 0.90.3
>            Reporter: zhoushuaifeng
>             Fix For: 0.90.4
>
>         Attachments: HBASE-3969-solution1-for-branch.patch, HBASE-3969-solution1.patch
>
>
> Compaction checker will send regions to the compact queue to do compact. But the priority
of these regions is too low if these regions have only a few storefiles. When there is large
through output, and the compact queue will aways have some regions with higher priority. This
may causing the major compact be delayed for a long time(even a few days),  and outdated data
cleaning will also be delayed.
> In our test case, we found some regions sent to the queue by major compact checker hunging
in the queue for more than 2 days! Some scanners on these regions cannot get availably data
for a long time and lease expired.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message