hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14708) Use copy on write TreeMap for region location cache
Date Tue, 27 Oct 2015 22:21:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977275#comment-14977275
] 

Hadoop QA commented on HBASE-14708:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12769079/HBASE-14708.patch
  against master branch at commit d5d81d675ace2d87c4ac19562b6b0a29da3d8902.
  ATTACHMENT ID: 12769079

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:red}-1 javac{color}.  The patch appears to cause mvn compile goal to fail with
Hadoop version 2.4.0.

    Compilation errors resume:
    [ERROR] COMPILATION ERROR : 
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/types/CopyOnWriteTreeMap.java:[241,27]
cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/types/CopyOnWriteTreeMap.java:[249,33]
no suitable method found for remove(java.lang.Object,java.lang.Object)
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/types/CopyOnWriteTreeMap.java:[257,33]
cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/types/CopyOnWriteTreeMap.java:[265,27]
cannot find symbol
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile
(default-compile) on project hbase-common: Compilation failure: Compilation failure:
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/types/CopyOnWriteTreeMap.java:[241,27]
cannot find symbol
[ERROR] symbol:   method putIfAbsent(K,V)
[ERROR] location: variable newMap of type java.util.TreeMap<K,V>
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/types/CopyOnWriteTreeMap.java:[249,33]
no suitable method found for remove(java.lang.Object,java.lang.Object)
[ERROR] method java.util.TreeMap.remove(java.lang.Object) is not applicable
[ERROR] (actual and formal argument lists differ in length)
[ERROR] method java.util.AbstractMap.remove(java.lang.Object) is not applicable
[ERROR] (actual and formal argument lists differ in length)
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/types/CopyOnWriteTreeMap.java:[257,33]
cannot find symbol
[ERROR] symbol:   method replace(K,V,V)
[ERROR] location: variable newMap of type java.util.TreeMap<K,V>
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/types/CopyOnWriteTreeMap.java:[265,27]
cannot find symbol
[ERROR] symbol:   method replace(K,V)
[ERROR] location: variable newMap of type java.util.TreeMap<K,V>
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following
articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hbase-common
    

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16250//console

This message is automatically generated.

> Use copy on write TreeMap for region location cache
> ---------------------------------------------------
>
>                 Key: HBASE-14708
>                 URL: https://issues.apache.org/jira/browse/HBASE-14708
>             Project: HBase
>          Issue Type: Improvement
>          Components: Client
>    Affects Versions: 1.1.2
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>            Priority: Critical
>             Fix For: 2.0.0, 1.2.0, 1.3.0
>
>         Attachments: HBASE-14708.patch, location_cache_times.pdf, result.csv
>
>
> Internally a co-worker profiled their application that was talking to HBase. > 60%
of the time was spent in locating a region. This was while the cluster was stable and no regions
were moving.
> To figure out if there was a faster way to cache region location I wrote up a benchmark
here: https://github.com/elliottneilclark/benchmark-hbase-cache
> This tries to simulate a heavy load on the location cache. 
> * 24 different threads.
> * 2 Deleting location data
> * 2 Adding location data
> * Using floor to get the result.
> To repeat my work just run ./run.sh and it should produce a result.csv
> Results:
> ConcurrentSkiplistMap is a good middle ground. It's got equal speed for reading and writing.
> However most operations will not need to remove or add a region location. There will
be potentially several orders of magnitude more reads for cached locations than there will
be on clearing the cache.
> So I propose a copy on write tree map.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message