accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-1755) BatchWriter blocks all addMutation calls while binning mutations
Date Mon, 29 Feb 2016 14:39:18 GMT


ASF GitHub Bot commented on ACCUMULO-1755:

Github user dlmarion commented on the pull request:
    Updated based on Keiths concerns, changed implementation such that:
    1. MutationWriter uses a ThreadPoolExecutor with 1 thread instead of a Timer. We might
be able to change this later, not sure.
    2. The ThreadPoolExecutor uses a LinkedTransferQueue as its work queue
    3. MutationWriter.queueMutations attempts to put the MutationSet onto the work queue with
no timeout. If there is a consumer available, then the MutationSet will get put onto the work
queue and processed by a background thread. If no consumer is available (e.g. it's currently
busy), then the MutationSet will be binned in the current thread.
    I also changed some variables to Atomic objects and removed the synchronization modifier
from TSBW.updateBatchStats()

> BatchWriter blocks all addMutation calls while binning mutations
> ----------------------------------------------------------------
>                 Key: ACCUMULO-1755
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: client
>            Reporter: Adam Fuchs
>            Assignee: Dave Marion
>             Fix For: 1.8.0
>          Time Spent: 50m
>  Remaining Estimate: 0h
> Through code inspection, we found that the BatchWriter bins mutations inside of a synchronized
block that covers calls to addMutation. Binning potentially involves lookups of tablet metadata
and processes a fair amount of information. We will get better parallelism if we can either
unlock the lock while binning, dedicate another thread to do the binning, or use one of the
send threads to do the binning.
> This has not been verified empirically yet, so there is not yet any profiling info to
indicate the level of improvement that we should expect. Profiling and repeatable demonstration
of this performance bottleneck should be the first step on this ticket.

This message was sent by Atlassian JIRA

View raw message