accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Wall <mjw...@gmail.com>
Subject Re: tserver.compaction.*.concurrent.max behavior in Accumulo 1.8.1
Date Thu, 23 Mar 2017 14:27:39 GMT
Max,

On you 3 node cluster, how many tables are you ingesting into?  How many
tablets are in each table?  Are the tablets equally spread amongst the 3
tablet servers?

Mike

On Thu, Mar 23, 2017 at 10:13 AM Massimilian Mattetti <MASSIMIL@il.ibm.com>
wrote:

> With the configuration I presented before the concurrent major compactions
> are never more than 3 per tablet server while the minor are under the 4 per
> node. Can one of the other configurations be the cause of this behavior?
>
> Regards,
> Max
>
>
>
> From:        Dave Marion <dlmarion@comcast.net>
> To:        user@accumulo.apache.org, Massimilian Mattetti/Haifa/IBM@IBMIL
> Date:        23/03/2017 14:55
> Subject:        Re: tserver.compaction.*.concurrent.max behavior in
> Accumulo 1.8.1
> ------------------------------
>
>
>
> Can you explain more what you mean by "My problem is that both the minor
> and major compactions do not overcome their default max values?" I have
> done some testing with 1.8.1 and specifically modifying
> tserver.compaction.major.concurrent.max to a higher number and I have seen
> it take effect.
>
> On March 23, 2017 at 7:54 AM Massimilian Mattetti <MASSIMIL@il.ibm.com>
> wrote:
>
> Hi All,
>
> I am running some heavy ingestion process on a 3 nodes cluster of Accumulo
> 1.8.1, using the following configuration:
>
> table.compaction.minor.logs.threshold=10
> table.durability=flush
> table.file.max=30
>
> tserver.wal.blocksize=2G
> tserver.walog.max.size=4G
> tserver.mutation.queue.max=2M
> tserver.memory.maps.max=4G
> tserver.compaction.minor.concurrent.max=50
> tserver.compaction.major.concurrent.max=8
>
> My problem is that both the minor and major compactions do not overcome
> their default max values. I checked the config from the shell and it looks
> fine to me:
>
> default    | tserver.compaction.major.concurrent.max ................ | 3
> system    |                     @override
> ........................................... | 8
>
> default    | tserver.compaction.minor.concurrent.max ............... | 4
> system    |                     @override
> ........................................... | 50
>
> Is something changed from 1.8.0? I haven't seen such behavior with the
> previous version.
> Thanks.
>
> Regards,
> Max
>
>
>
>
>

Mime
View raw message