accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-4187) Rate limiting of major compactions
Date Mon, 18 Apr 2016 14:56:25 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15245808#comment-15245808
] 

ASF GitHub Bot commented on ACCUMULO-4187:
------------------------------------------

Github user joshelser commented on the pull request:

    https://github.com/apache/accumulo/pull/90#issuecomment-211414852
  
    I think everything I asked about has been taken care of. Thanks for that, Shawn.
    
    The only thing I don't see (and I didn't say it explicitly earlier, so I don't think it's
a blocker to merge this in) is a high-level test. I see you added some tests for the rate-limiter
piece. I'm wondering if we could make an integration test specifically to test this feature.
It's nice when we have a general test class (built around a minicluster) available so that
we can easily test potential bugs and add new tests easily.
    
    @keith-turner LMK if you have time to merge this in and run the tests. Otherwise, I'll
kick off something myself.


> Rate limiting of major compactions
> ----------------------------------
>
>                 Key: ACCUMULO-4187
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4187
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: core, tserver
>            Reporter: Shawn Walker
>            Assignee: Shawn Walker
>            Priority: Minor
>             Fix For: 1.8.0
>
>
> In discussing [ACCUMULO-4166] with Keith Turner, we decided that the underlying issue
is that major compactions can overwhelm a tablet server, rendering it nearly unresponsive.
> To address this, we should take a cue from Apache Cassandra and restrict how quickly
we perform major compactions.  Rate limiting reads and writes involved in major compactions
will directly affect the IO load caused by major compactions, and should also indirectly affect
the CPU load.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message