hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14708) Use copy on write TreeMap for region location cache
Date Wed, 04 Nov 2015 03:23:28 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988811#comment-14988811
] 

Hadoop QA commented on HBASE-14708:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12770491/HBASE-14708-v9.patch
  against master branch at commit 68717241040e6dad04958e49e5373887dd677dfd.
  ATTACHMENT ID: 12770491

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 8 new or modified
tests.

    {color:red}-1 javac{color}.  The patch appears to cause mvn compile goal to fail with
Hadoop version 2.4.0.

    Compilation errors resume:
    [ERROR] COMPILATION ERROR : 
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/types/CopyOnWriteArrayMap.java:[579,17]
org.apache.hadoop.hbase.types.CopyOnWriteArrayMap.ArrayKeyIterator is not abstract and does
not override abstract method remove() in java.util.Iterator
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/types/CopyOnWriteArrayMap.java:[600,17]
org.apache.hadoop.hbase.types.CopyOnWriteArrayMap.ArrayValueIterator is not abstract and does
not override abstract method remove() in java.util.Iterator
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile
(default-compile) on project hbase-common: Compilation failure: Compilation failure:
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/types/CopyOnWriteArrayMap.java:[579,17]
org.apache.hadoop.hbase.types.CopyOnWriteArrayMap.ArrayKeyIterator is not abstract and does
not override abstract method remove() in java.util.Iterator
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-common/src/main/java/org/apache/hadoop/hbase/types/CopyOnWriteArrayMap.java:[600,17]
org.apache.hadoop.hbase.types.CopyOnWriteArrayMap.ArrayValueIterator is not abstract and does
not override abstract method remove() in java.util.Iterator
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following
articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hbase-common
    

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/16376//console

This message is automatically generated.

> Use copy on write TreeMap for region location cache
> ---------------------------------------------------
>
>                 Key: HBASE-14708
>                 URL: https://issues.apache.org/jira/browse/HBASE-14708
>             Project: HBase
>          Issue Type: Improvement
>          Components: Client
>    Affects Versions: 1.1.2
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>            Priority: Critical
>             Fix For: 2.0.0, 1.2.0, 1.3.0
>
>         Attachments: HBASE-14708-v2.patch, HBASE-14708-v3.patch, HBASE-14708-v4.patch,
HBASE-14708-v5.patch, HBASE-14708-v6.patch, HBASE-14708-v7.patch, HBASE-14708-v8.patch, HBASE-14708-v9.patch,
HBASE-14708.patch, anotherbench.zip, location_cache_times.pdf, result.csv
>
>
> Internally a co-worker profiled their application that was talking to HBase. > 60%
of the time was spent in locating a region. This was while the cluster was stable and no regions
were moving.
> To figure out if there was a faster way to cache region location I wrote up a benchmark
here: https://github.com/elliottneilclark/benchmark-hbase-cache
> This tries to simulate a heavy load on the location cache. 
> * 24 different threads.
> * 2 Deleting location data
> * 2 Adding location data
> * Using floor to get the result.
> To repeat my work just run ./run.sh and it should produce a result.csv
> Results:
> ConcurrentSkiplistMap is a good middle ground. It's got equal speed for reading and writing.
> However most operations will not need to remove or add a region location. There will
be potentially several orders of magnitude more reads for cached locations than there will
be on clearing the cache.
> So I propose a copy on write tree map.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message