accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-4112) Poor performance due to MinC start/stop updates are always hsync'd
Date Wed, 09 Mar 2016 15:23:40 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187237#comment-15187237
] 

ASF GitHub Bot commented on ACCUMULO-4112:
------------------------------------------

Github user keith-turner commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/79#discussion_r55533890
  
    --- Diff: server/tserver/src/main/java/org/apache/accumulo/tserver/TabletServer.java ---
    @@ -2966,13 +2966,26 @@ public Void run() {
         }
       }
     
    +  private Durability getMincEventDurability(KeyExtent extent) {
    +    TableConfiguration conf;
    +    if (extent.isMeta() || extent.isRootTablet()) {
    --- End diff --
    
    > Is there some other setting for user tables that's more appropriate? 
    
    This is the best setting I know of... Since the walog minc entries are used during recovery,
its nice to have them at the same sync level as the tablets metadata.
    
    > so the || extent.isRootTablet() is redundant
    
    Nice catch, I wrongly assumed isMeta() was a more narrow check.  I used to know it did
a wider check, but had forgotten.  
    
    >  I think refactoring KeyExtent#isMeta is more appropriate
    
    Changing its name would need to be a separate PR, because it would be very noisy drowning
out the changes for 4112.  I agree another name that implies more than `accumulo.metadata`
would be good, however I am not sure what that name would be.


> Poor performance due to MinC start/stop updates are always hsync'd
> ------------------------------------------------------------------
>
>                 Key: ACCUMULO-4112
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4112
>             Project: Accumulo
>          Issue Type: Bug
>          Components: tserver
>         Environment: Fluo testing on a 20-node cluster
>            Reporter: Eric Newton
>            Assignee: Keith Turner
>            Priority: Critical
>             Fix For: 1.7.2, 1.8.0
>
>         Attachments: HSyncOverheadExperiment.png, MinCFlushPerfTest.java, Sync-Flush-Log-Performance.png
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> [~kturner] writes:
> {quote}
> I was running a Fluo test with 1.8.0-SNAP on my workstation.  My Fluo table had a ton
of tablets.   I was seeing terrible performance.   I started looking at the tserver and noticed
it was always calling hsync.  I tracked down the problem to the fact that when minc start
and stop events are written to the log they are always written w/ sync level.   My poor little
tserver was constantly minor compacting (probably had around 600 tablets that were all being
written to).  
> I changed the test config to create like 15 tablets and the performance was much better.
 All cores were 100% utilized, which was not the case when hsync was always called.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message