accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Massimilian Mattetti" <MASSI...@il.ibm.com>
Subject Re: tserver.compaction.*.concurrent.max behavior in Accumulo 1.8.1
Date Thu, 23 Mar 2017 14:37:16 GMT
One table, 9 pre-split tablets, 3 tablets per server and the data is 
uniform distributed among each tablet.
Max




From:   Michael Wall <mjwall@gmail.com>
To:     user@accumulo.apache.org
Date:   23/03/2017 16:28
Subject:        Re: tserver.compaction.*.concurrent.max behavior in 
Accumulo 1.8.1



Max,

On you 3 node cluster, how many tables are you ingesting into?  How many 
tablets are in each table?  Are the tablets equally spread amongst the 3 
tablet servers?

Mike

On Thu, Mar 23, 2017 at 10:13 AM Massimilian Mattetti <MASSIMIL@il.ibm.com
> wrote:
With the configuration I presented before the concurrent major compactions 
are never more than 3 per tablet server while the minor are under the 4 
per node. Can one of the other configurations be the cause of this 
behavior?

Regards,
Max



From:        Dave Marion <dlmarion@comcast.net>
To:        user@accumulo.apache.org, Massimilian Mattetti/Haifa/IBM@IBMIL
Date:        23/03/2017 14:55
Subject:        Re: tserver.compaction.*.concurrent.max behavior in 
Accumulo 1.8.1



Can you explain more what you mean by "My problem is that both the minor 
and major compactions do not overcome their default max values?" I have 
done some testing with 1.8.1 and specifically modifying 
tserver.compaction.major.concurrent.max to a higher number and I have seen 
it take effect.

On March 23, 2017 at 7:54 AM Massimilian Mattetti <MASSIMIL@il.ibm.com> 
wrote:

Hi All,

I am running some heavy ingestion process on a 3 nodes cluster of Accumulo 
1.8.1, using the following configuration:

table.compaction.minor.logs.threshold=10
table.durability=flush
table.file.max=30

tserver.wal.blocksize=2G
tserver.walog.max.size=4G
tserver.mutation.queue.max=2M
tserver.memory.maps.max=4G
tserver.compaction.minor.concurrent.max=50
tserver.compaction.major.concurrent.max=8

My problem is that both the minor and major compactions do not overcome 
their default max values. I checked the config from the shell and it looks 
fine to me:

default    | tserver.compaction.major.concurrent.max ................ | 3
system    |                     @override 
........................................... | 8

default    | tserver.compaction.minor.concurrent.max ............... | 4
system    |                     @override 
........................................... | 50

Is something changed from 1.8.0? I haven't seen such behavior with the 
previous version. 
Thanks.

Regards,
Max








Mime
View raw message