Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@accumulo.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <CAP6MKstzWeo25MD-_6OBHJu4OMFRQhJUB-nf0x6pM9ag2qi9rQ@mail.gmail.com>
References: 
 <CAP6MKstzWeo25MD-_6OBHJu4OMFRQhJUB-nf0x6pM9ag2qi9rQ@mail.gmail.com>
Date: Thu, 28 Feb 2013 09:26:06 -0500
Message-ID: 
 <CAGUtCHqJyVOh04txxOYNQ-HigRKqR4so43vVn4_oqdOzWx4kSA@mail.gmail.com>
Subject: Re: Throttle major compaction.
From: Keith Turner <keith@deenlo.com>
To: user@accumulo.apache.org
Content-Type: text/plain; charset=ISO-8859-1

I thnk this could be done fairly easily with an iterator.  An iterator
configured for major compaction could rate limit data.

On Thu, Feb 28, 2013 at 8:27 AM, Denis <denis@camfex.cz> wrote:
> Hi.
>
> Major compaction loads hard disks very hard, even with
> tserver.compaction.major.concurrent.max=1
>
> Besides nice peaks on the Load Average and IOstat graps during major
> compaction, such high load also badly affects query performance and
> latency.
>
> HBase and Cassandra both have settings to throttle major compaction
> (Cassandra's default is 16 Mb/s which is about 1/3 of the peek values
> I see during Accumulo's major compaction)
>
> But it looks like Accumulo does not have such setting and I have not
> found relevant questions in the bugtracker and maillist archive.
>
> How do you deal with the major compaction and its performance effects?