accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Medinets <david.medin...@gmail.com>
Subject Re: How to speed up table compaction [SEC=UNOFFICIAL]
Date Tue, 06 May 2014 15:52:18 GMT
Can that property be changed on the fly? And Snappy needs to be installed
throughout the cluster before making the configuration change?


On Tue, May 6, 2014 at 12:43 AM, Josh Elser <josh.elser@gmail.com> wrote:

> Depending on the CPU/IO ratio for your system, switching to a different
> compression codec might help. Snappy tends to be a bit quicker writing out
> data as opposed to gzip at the cost of being larger on disk. The increase
> in final size on disk might be prohibitive depending on your requirements
> though.
>
> I forget the table property off hand, but, if you haven't changed this
> already, it will be the property with a defauly value of 'gz' :)
> On May 5, 2014 6:43 PM, "Dickson, Matt MR" <matt.dickson@defence.gov.au>
> wrote:
>
>>  *UNOFFICIAL*
>> I'm trying to compact a table and have had a queue of 60,000 compactions
>> with only 800 running at a time.  This has now been running for 4 days and
>> only decreased the queue by 8,000.
>>
>> I've increased tserver.compaction.major.concurrent.max=12 and stopped all
>> ingest but not seen a change in progress.  Are there other accumulo
>> settings I can alter to improve this?  I also saw
>> tserver.compaction.major.thread.files.open.max=10 should this be increased?
>>
>> Thanks in advance,
>> Matt
>>
>

Mime
View raw message