hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2457) RS gets stuck compacting region ad infinitum
Date Fri, 16 Apr 2010 05:53:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12857683#action_12857683
] 

stack commented on HBASE-2457:
------------------------------

Here is the code:

{code}
      if (!majorcompaction && !references) {
        // Here we select files for incremental compaction.  
        // The rule is: if the largest(oldest) one is more than twice the 
        // size of the second, skip the largest, and continue to next...,
        // until we meet the compactionThreshold limit.
        for (point = 0; point < countOfFiles - 1; point++) {
          if ((fileSizes[point] < fileSizes[point + 1] * 2) && 
               (countOfFiles - point) <= maxFilesToCompact) {
            break;
          }
          skipped += fileSizes[point];
        }
        filesToCompact = new ArrayList<StoreFile>(filesToCompact.subList(point,
          countOfFiles));
...
{code}

Todd did a listing of the Store:

{code}
-rw-r--r--   3 todd supergroup  134676288 2010-04-15 20:58 /hbase/test1/1736416594/actions/7876304170849844721
-rw-r--r--   3 todd supergroup   61309324 2010-04-15 20:59 /hbase/test1/1736416594/actions/3289606064411356106
-rw-r--r--   3 todd supergroup   24381446 2010-04-15 21:00 /hbase/test1/1736416594/actions/4995952634622872563
-rw-r--r--   3 todd supergroup    1194376 2010-04-15 21:01 /hbase/test1/1736416594/actions/3688488928995595533
-rw-r--r--   3 todd supergroup    2712677 2010-04-15 21:01 /hbase/test1/1736416594/actions/5274234463019618354
-rw-r--r--   3 todd supergroup     532824 2010-04-15 21:01 /hbase/test1/1736416594/actions/5321716733066884905
-rw-r--r--   3 todd supergroup   12103493 2010-04-15 21:01 /hbase/test1/1736416594/actions/6876448782185582993
-rw-r--r--   3 todd supergroup    5855317 2010-04-15 21:01 /hbase/test1/1736416594/actions/7381870009831588255
{code}

If you trace, the olders is > 2 * the next oldest, and so on.  Some of the times are same
so not sure how that plays out.... (the above is not strictly ordered ... those of same time
may not be proper chronological order).



> RS gets stuck compacting region ad infinitum
> --------------------------------------------
>
>                 Key: HBASE-2457
>                 URL: https://issues.apache.org/jira/browse/HBASE-2457
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.4
>            Reporter: Todd Lipcon
>            Priority: Critical
>         Attachments: log.gz, stack
>
>
> Testing 0.20_pre_durability@934643, I ended up in a state where one region server got
stuck compacting a single region over and over again forever. This was with a special config
with very low flush threshold in order to stress test flush/compact code.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message