Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Fri, 26 Feb 2016 22:31:18 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.12672651.1381167344000.160797.1456525878197@Atlassian.JIRA>
In-Reply-To: <JIRA.12672651.1381167344000@Atlassian.JIRA>
References: <JIRA.12672651.1381167344000@Atlassian.JIRA>
 <JIRA.12672651.1381167344040@arcas>
Subject: [jira] [Commented] (ACCUMULO-1755) BatchWriter blocks all
 addMutation calls while binning mutations
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/ACCUMULO-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15169973#comment-15169973 ] 

ASF GitHub Bot commented on ACCUMULO-1755:
------------------------------------------

Github user joshelser commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/75#discussion_r54311365
  
    --- Diff: core/src/main/java/org/apache/accumulo/core/client/impl/TabletServerBatchWriter.java ---
    @@ -699,7 +701,16 @@ else if (Tables.getTableState(context.getInstance(), table) == TableState.OFFLIN
     
         }
     
    -    void addMutations(MutationSet mutationsToSend) {
    +    void queueMutations(final MutationSet mutationsToSend) {
    +      queueThreadPool.submit(new Runnable() {
    --- End diff --
    
    Ultimately, I really don't have that strong of an opinion here. I think using the ExecutorService as a queue for MutationSets is a smell (instead of directly maintaining such a queue), but I can understand the argument for using the ExecutorService *because* it encompasses both approaches. I don't think there's a big reason one way or the other; I simply think that the pseudo-code outline above is clearer than Dave's current changes. If Keith and Dave are happy, then merge it and move on.


> BatchWriter blocks all addMutation calls while binning mutations
> ----------------------------------------------------------------
>
>                 Key: ACCUMULO-1755
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1755
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: client
>            Reporter: Adam Fuchs
>            Assignee: Dave Marion
>             Fix For: 1.8.0
>
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Through code inspection, we found that the BatchWriter bins mutations inside of a synchronized block that covers calls to addMutation. Binning potentially involves lookups of tablet metadata and processes a fair amount of information. We will get better parallelism if we can either unlock the lock while binning, dedicate another thread to do the binning, or use one of the send threads to do the binning.
> This has not been verified empirically yet, so there is not yet any profiling info to indicate the level of improvement that we should expect. Profiling and repeatable demonstration of this performance bottleneck should be the first step on this ticket.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)