hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-1813) Remove compaction governor
Date Thu, 03 Sep 2009 05:28:32 GMT
Remove compaction governor

                 Key: HBASE-1813
                 URL: https://issues.apache.org/jira/browse/HBASE-1813
             Project: Hadoop HBase
          Issue Type: Bug
            Reporter: stack
             Fix For: 0.21.0

It might make sense now undoing the compaction governor:

>From the list (read in reverse):

Andrew Purtell
 to hbase-user
show details 10:18 PM (7 minutes ago)
I had a similar thought, that this could be removed in 0.21.

  - Andy

From: stack <stack@duboce.net>
To: hbase-user@hadoop.apache.org
Sent: Wednesday, September 2, 2009 8:13:06 PM
Subject: Re: Compactions no longer limited


Maybe now that in 0.20.0 we only run compaction on open IFF the region has
references, may be this facility is no longer needed?


On Wed, Sep 2, 2009 at 6:05 PM, Andrew Purtell <apurtell@apache.org> wrote:

> Hi Ken,
> Compactions are serviced by a thread which sleeps for a configurable
> interval and then wakes to do work. As compaction requests are raised, they
> are queued and the thread is signaled and wakes early. When a region server
> first starts up, a limit is imposed on how many compaction requests can be
> serviced from the queue before the thread goes back to sleep. The limit is
> gradually relaxed. The net effect is to draw out compactions over an initial
> "safe mode". The reason this is done is to limit load on DFS when the
> cluster is starting. Compactions during this period often process flushes
> from the last shutdown and can well trigger splits. Around the 0.19
> timeframe I had a 25 node test cluster pushing 1000 regions per node.
> Startup compaction/split activity would crush DFS and prevent successful
> (re)starts. The mechanism described here solved that issue.
>    - Andy

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message