accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis <de...@camfex.cz>
Subject Re: Throttle major compaction.
Date Thu, 28 Feb 2013 15:03:42 GMT
Hi, Keith

Thanks for the hint to use iterator for that.

Although this approach can help in my case, it may be tricky to use it
in other cases, because (as far as I understood), with iterator one
can throttle the uncompressed data and actual throttling of the hard
disk operations would depend on the data compressibility in the tablet
being compacted and can vary widely.

I opened the ticket ACCUMULO-1128.

On 2/28/13, Keith Turner <keith@deenlo.com> wrote:
> Can you open a ticket for this?  We could post the iterator as a patch
> to that ticket.  Then people could use that patch.
>
> On Thu, Feb 28, 2013 at 8:27 AM, Denis <denis@camfex.cz> wrote:
>> Hi.
>>
>> Major compaction loads hard disks very hard, even with
>> tserver.compaction.major.concurrent.max=1
>>
>> Besides nice peaks on the Load Average and IOstat graps during major
>> compaction, such high load also badly affects query performance and
>> latency.
>>
>> HBase and Cassandra both have settings to throttle major compaction
>> (Cassandra's default is 16 Mb/s which is about 1/3 of the peek values
>> I see during Accumulo's major compaction)
>>
>> But it looks like Accumulo does not have such setting and I have not
>> found relevant questions in the bugtracker and maillist archive.
>>
>> How do you deal with the major compaction and its performance effects?
>

Mime
View raw message