hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Pearson (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-2615) Add max number of mapfiles to compact at one time giveing up a minor & major compaction
Date Tue, 15 Jan 2008 21:51:34 GMT
Add max number of mapfiles to compact at one time giveing up a minor & major compaction
---------------------------------------------------------------------------------------

                 Key: HADOOP-2615
                 URL: https://issues.apache.org/jira/browse/HADOOP-2615
             Project: Hadoop
          Issue Type: Improvement
            Reporter: Billy Pearson
            Priority: Minor
             Fix For: 0.17.0


Currently we do compaction on a region when the hbase.hstore.compactionThreshold is reached
- default 3

I thank we should configure a max number of mapfiles to compact at one time simulator to doing
a minor compaction in bigtable. This keep compaction's form getting tied up in one region
to long letting other regions get way to many memcache flushes making compaction take longer
and longer for each region

If we did that when a regions updates start to slack off the max number will eventuly include
all mapfiles causeing a major compaction on that region. Unlike big table this would leave
the master out of the process and letting the region server handle the major compaction when
it has time.

When doing a minor compaction on a few files I thank we should compact the newest mapfiles
first leave the larger/older ones for when we have low updates to a region.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message