accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-1755) BatchWriter blocks all addMutation calls while binning mutations
Date Fri, 26 Feb 2016 16:36:18 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15169278#comment-15169278
] 

ASF GitHub Bot commented on ACCUMULO-1755:
------------------------------------------

GitHub user dlmarion opened a pull request:

    https://github.com/apache/accumulo/pull/75

    ACCUMULO-1755: Modified TSBW so that all client threads will not bloc…

    I ditched review board and went back to the drawing board.
    
    Before this change all client threads that were adding mutations to a BatchWriter would
block
    when the mutations needed to be written to the tablet servers. This change gives each
client
    thread their own MutationWriter object using a ThreadLocal. With this change only one
client
    should block on adding the mutations to the sendThreadPool object, and allows the other
client
    threads to push mutations onto a new MutationSet.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/apache/accumulo ACCUMULO-1755

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/accumulo/pull/75.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #75
    
----
commit 9bd48cdbd25a7baf19bbda007a8e36f8ec81cf07
Author: Dave Marion <dlmarion@apache.org>
Date:   2016-02-26T16:26:53Z

    ACCUMULO-1755: Modified TSBW so that all client threads will not block on binMutations
    
    Before this change all client threads that were adding mutations to a BatchWriter would
block
    when the mutations needed to be written to the tablet servers. This change gives each
client
    thread their own MutationWriter object using a ThreadLocal. With this change only one
client
    should block on adding the mutations to the sendThreadPool object, and allows the other
client
    threads to push mutations onto a new MutationSet.

----


> BatchWriter blocks all addMutation calls while binning mutations
> ----------------------------------------------------------------
>
>                 Key: ACCUMULO-1755
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1755
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: client
>            Reporter: Adam Fuchs
>            Assignee: Dave Marion
>             Fix For: 1.8.0
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Through code inspection, we found that the BatchWriter bins mutations inside of a synchronized
block that covers calls to addMutation. Binning potentially involves lookups of tablet metadata
and processes a fair amount of information. We will get better parallelism if we can either
unlock the lock while binning, dedicate another thread to do the binning, or use one of the
send threads to do the binning.
> This has not been verified empirically yet, so there is not yet any profiling info to
indicate the level of improvement that we should expect. Profiling and repeatable demonstration
of this performance bottleneck should be the first step on this ticket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message