accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: Throttle major compaction.
Date Thu, 28 Feb 2013 15:16:48 GMT
On Thu, Feb 28, 2013 at 10:03 AM, Denis <denis@camfex.cz> wrote:
> Hi, Keith
>
> Thanks for the hint to use iterator for that.
>
> Although this approach can help in my case, it may be tricky to use it
> in other cases, because (as far as I understood), with iterator one
> can throttle the uncompressed data and actual throttling of the hard
> disk operations would depend on the data compressibility in the tablet
> being compacted and can vary widely.

Thats a good point.  I posted this on the ticket to move the discussion there.

>
> I opened the ticket ACCUMULO-1128.
>
> On 2/28/13, Keith Turner <keith@deenlo.com> wrote:
>> Can you open a ticket for this?  We could post the iterator as a patch
>> to that ticket.  Then people could use that patch.
>>
>> On Thu, Feb 28, 2013 at 8:27 AM, Denis <denis@camfex.cz> wrote:
>>> Hi.
>>>
>>> Major compaction loads hard disks very hard, even with
>>> tserver.compaction.major.concurrent.max=1
>>>
>>> Besides nice peaks on the Load Average and IOstat graps during major
>>> compaction, such high load also badly affects query performance and
>>> latency.
>>>
>>> HBase and Cassandra both have settings to throttle major compaction
>>> (Cassandra's default is 16 Mb/s which is about 1/3 of the peek values
>>> I see during Accumulo's major compaction)
>>>
>>> But it looks like Accumulo does not have such setting and I have not
>>> found relevant questions in the bugtracker and maillist archive.
>>>
>>> How do you deal with the major compaction and its performance effects?
>>

Mime
View raw message