accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: How to speed up table compaction [SEC=UNOFFICIAL]
Date Tue, 06 May 2014 04:43:32 GMT
Depending on the CPU/IO ratio for your system, switching to a different
compression codec might help. Snappy tends to be a bit quicker writing out
data as opposed to gzip at the cost of being larger on disk. The increase
in final size on disk might be prohibitive depending on your requirements
though.

I forget the table property off hand, but, if you haven't changed this
already, it will be the property with a defauly value of 'gz' :)
On May 5, 2014 6:43 PM, "Dickson, Matt MR" <matt.dickson@defence.gov.au>
wrote:

>  *UNOFFICIAL*
> I'm trying to compact a table and have had a queue of 60,000 compactions
> with only 800 running at a time.  This has now been running for 4 days and
> only decreased the queue by 8,000.
>
> I've increased tserver.compaction.major.concurrent.max=12 and stopped all
> ingest but not seen a change in progress.  Are there other accumulo
> settings I can alter to improve this?  I also saw
> tserver.compaction.major.thread.files.open.max=10 should this be increased?
>
> Thanks in advance,
> Matt
>

Mime
View raw message