hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "huaxiang sun (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-17172) Optimize major mob compaction with _del files
Date Wed, 23 Nov 2016 18:35:58 GMT
huaxiang sun created HBASE-17172:

             Summary: Optimize major mob compaction with _del files
                 Key: HBASE-17172
                 URL: https://issues.apache.org/jira/browse/HBASE-17172
             Project: HBase
          Issue Type: Improvement
          Components: mob
    Affects Versions: 2.0.0
            Reporter: huaxiang sun
            Assignee: huaxiang sun

Today, when there is a _del file in mobdir, with major mob compaction, every mob file will
be recompacted, this causes lots of IO and slow down major mob compaction (may take months
to finish). This needs to be improved. A few ideas are: 

1) Do not compact all _del files into one, instead, compact them based on groups with startKey
as the key. Then use firstKey/startKey to make each mob file to see if the _del file needs
to be included for this partition.

2). Based on the timerange of the _del file, compaction for files after that timerange does
not need to include the _del file as these are newer files.

This message was sent by Atlassian JIRA

View raw message