hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Pearson (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-834) Upper bound on files we compact at any one time
Date Sat, 16 Aug 2008 06:33:44 GMT

     [ https://issues.apache.org/jira/browse/HBASE-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Billy Pearson updated HBASE-834:

    Affects Version/s: 0.3.0
               Status: Patch Available  (was: Open)

I thank I got this working correctly I test on my end and it all works ok
Sense we where doing the minor compaction (incremental compaction) on the HStore level I did
the same for the Major Compaction
I set the default to 1 day fill free to change that to what you guys thank is a correct default.

2008-08-15 23:34:37,626 INFO org.apache.hadoop.hbase.regionserver.HRegion: starting compaction
on region -ROOT-,,0
2008-08-15 23:34:37,626 DEBUG org.apache.hadoop.hbase.regionserver.HLog: changing sequence
number from 0 to 866762
2008-08-15 23:34:37,634 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Major compaction
triggered on store: 70236052/info
2008-08-15 23:34:37,682 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction
of 1 files into /hbase/-ROOT-/compaction.dir/70236052/info/mapfiles/5245590111629292638
2008-08-15 23:34:37,819 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/-ROOT-/compaction.dir/70236052/info/mapfiles/5245590111629292638
to /hbase/-ROOT-/70236052/info/mapfiles/8511958703098935844
2008-08-15 23:34:37,885 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction
of 70236052/info store size is 809.0
2008-08-15 23:34:37,890 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed
on region -ROOT-,,0 in 0sec

Some of my debug code I removed from the patch outputted the timestamps and folder location
of the lowTimestamp files. So I could make sure we where checking the correct folder and the
timestamps from the files where in mills and everything showed up correctly in the right format.

Please review and let me know.

> Upper bound on files we compact at any one time
> -----------------------------------------------
>                 Key: HBASE-834
>                 URL: https://issues.apache.org/jira/browse/HBASE-834
>             Project: Hadoop HBase
>          Issue Type: Improvement
>    Affects Versions: 0.2.1, 0.3.0
>            Reporter: stack
>            Assignee: Billy Pearson
>            Priority: Minor
>             Fix For: 0.2.1, 0.3.0
> From Billy in HBASE-64, which we closed because it got pulled all over the place:
> {code}
> Currently we do compaction on a region when the hbase.hstore.compactionThreshold is reached
- default 3
> I thank we should configure a max number of mapfiles to compact at one time simulator
to doing a minor compaction in bigtable. This keep compaction's form getting tied up in one
region to long letting other regions get way to many memcache flushes making compaction take
longer and longer for each region
> If we did that when a regions updates start to slack off the max number will eventuly
include all mapfiles causeing a major compaction on that region. Unlike big table this would
leave the master out of the process and letting the region server handle the major compaction
when it has time.
> When doing a minor compaction on a few files I thank we should compact the newest mapfiles
first leave the larger/older ones for when we have low updates to a region.
> {code}

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message