accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Wall <mjw...@gmail.com>
Subject Re: tserver.compaction.*.concurrent.max behavior in Accumulo 1.8.1
Date Thu, 23 Mar 2017 14:43:36 GMT
Max,

So your max major compactions will be 3 per tablet server.  Accumulo will
not run 2 majors on the same tablet concurrently.

Mike

On Thu, Mar 23, 2017 at 10:37 AM Massimilian Mattetti <MASSIMIL@il.ibm.com>
wrote:

> One table, 9 pre-split tablets, 3 tablets per server and the data is
> uniform distributed among each tablet.
> Max
>
>
>
>
> From:        Michael Wall <mjwall@gmail.com>
> To:        user@accumulo.apache.org
> Date:        23/03/2017 16:28
> Subject:        Re: tserver.compaction.*.concurrent.max behavior in
> Accumulo 1.8.1
> ------------------------------
>
>
>
> Max,
>
> On you 3 node cluster, how many tables are you ingesting into?  How many
> tablets are in each table?  Are the tablets equally spread amongst the 3
> tablet servers?
>
> Mike
>
> On Thu, Mar 23, 2017 at 10:13 AM Massimilian Mattetti <
> *MASSIMIL@il.ibm.com* <MASSIMIL@il.ibm.com>> wrote:
> With the configuration I presented before the concurrent major compactions
> are never more than 3 per tablet server while the minor are under the 4 per
> node. Can one of the other configurations be the cause of this behavior?
>
> Regards,
> Max
>
>
>
> From:        Dave Marion <*dlmarion@comcast.net* <dlmarion@comcast.net>>
> To:        *user@accumulo.apache.org* <user@accumulo.apache.org>,
> Massimilian Mattetti/Haifa/IBM@IBMIL
> Date:        23/03/2017 14:55
> Subject:        Re: tserver.compaction.*.concurrent.max behavior in
> Accumulo 1.8.1
> ------------------------------
>
>
>
> Can you explain more what you mean by "My problem is that both the minor
> and major compactions do not overcome their default max values?" I have
> done some testing with 1.8.1 and specifically modifying
> tserver.compaction.major.concurrent.max to a higher number and I have seen
> it take effect.
>
> On March 23, 2017 at 7:54 AM Massimilian Mattetti <*MASSIMIL@il.ibm.com*
> <MASSIMIL@il.ibm.com>> wrote:
>
> Hi All,
>
> I am running some heavy ingestion process on a 3 nodes cluster of Accumulo
> 1.8.1, using the following configuration:
>
> table.compaction.minor.logs.threshold=10
> table.durability=flush
> table.file.max=30
>
> tserver.wal.blocksize=2G
> tserver.walog.max.size=4G
> tserver.mutation.queue.max=2M
> tserver.memory.maps.max=4G
> tserver.compaction.minor.concurrent.max=50
> tserver.compaction.major.concurrent.max=8
>
> My problem is that both the minor and major compactions do not overcome
> their default max values. I checked the config from the shell and it looks
> fine to me:
>
> default    | tserver.compaction.major.concurrent.max ................ | 3
> system    |                     @override
> ........................................... | 8
>
> default    | tserver.compaction.minor.concurrent.max ............... | 4
> system    |                     @override
> ........................................... | 50
>
> Is something changed from 1.8.0? I haven't seen such behavior with the
> previous version.
> Thanks.
>
> Regards,
> Max
>
>
>
>
>
>
>

Mime
View raw message