hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-938) major compaction period is not checked periodically
Date Fri, 31 Oct 2008 22:19:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644445#action_12644445
] 

stack commented on HBASE-938:
-----------------------------

I need to look into this but I just noticed how Billy made sure we never compact more than
often than the major compaction period, even if restarts in between:

{code}
        long lowTimestamp = getLowestTimestamp(fs, mapdir);
        lastMajorCompaction = System.currentTimeMillis() - lowTimestamp;
        if (lowTimestamp < (System.currentTimeMillis() - majorCompactionTime) &&
            lowTimestamp > 0l) {
          if (LOG.isDebugEnabled()) {
            LOG.debug("Major compaction triggered on store: " +
              this.storeNameStr + ". Time since last major compaction: " +
              ((System.currentTimeMillis() - lowTimestamp)/1000) + " seconds");
          }
...
{code}

We look at file timestamps and if oldest file was written > major compaction period ago,
then we run major compaction next time we check compactions.

So, for this issue, I need to confirm we only compact on a period and two, since optional
flush was removed when we implemented appends, needs to be a thread or something that runs
on a period looking to see if any regions in needs of major compaction.

> major compaction period is not checked periodically
> ---------------------------------------------------
>
>                 Key: HBASE-938
>                 URL: https://issues.apache.org/jira/browse/HBASE-938
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 0.18.0, 0.18.1
>         Environment: HBase 0.18 branch (should be RC1) + Hadoop 0.18 branch
>            Reporter: Rong-En Fan
>            Priority: Critical
>             Fix For: 0.19.0
>
>
> The major compaction period, hbase.hregion.majorcompaction, is not checked periodically.
Currently, we only request major compaction when the region is open or split at which point
we check whether the major compaction period is due.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message