hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17605) Refactor procedure framework code
Date Sat, 11 Feb 2017 07:16:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862278#comment-15862278
] 

Hadoop QA commented on HBASE-17605:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 23s {color} | {color:blue}
Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} |
{color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color}
| {color:green} The patch appears to include 1 new or modified test files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 33s {color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 29s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s {color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 19s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 23s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 3s {color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s {color} |
{color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s {color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 34s {color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 34s {color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 34s {color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 20s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 22s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 24s {color} | {color:red}
The patch causes 22 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 39s {color} | {color:red}
The patch causes 22 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 55s {color} | {color:red}
The patch causes 22 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 10s {color} | {color:red}
The patch causes 22 errors with Hadoop v2.6.4. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 25s {color} | {color:red}
The patch causes 22 errors with Hadoop v2.6.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m 40s {color} | {color:red}
The patch causes 22 errors with Hadoop v2.7.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m 53s {color} | {color:red}
The patch causes 22 errors with Hadoop v2.7.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 10m 8s {color} | {color:red}
The patch causes 22 errors with Hadoop v2.7.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 11m 22s {color} | {color:red}
The patch causes 22 errors with Hadoop v3.0.0-alpha2. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 22s {color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 42s {color} | {color:green}
hbase-procedure in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 34s {color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s {color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 52s {color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12852185/HBASE-17605.master.008.patch
|
| JIRA Issue | HBASE-17605 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  hbaseanti  checkstyle
 compile  |
| uname | Linux 98c3a3d021dd 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a05abd8 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
| mvninstall | https://builds.apache.org/job/PreCommit-HBASE-Build/5675/artifact/patchprocess/patch-mvninstall-hbase-server.txt
|
| compile | https://builds.apache.org/job/PreCommit-HBASE-Build/5675/artifact/patchprocess/patch-compile-hbase-server.txt
|
| javac | https://builds.apache.org/job/PreCommit-HBASE-Build/5675/artifact/patchprocess/patch-compile-hbase-server.txt
|
| findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/5675/artifact/patchprocess/patch-findbugs-hbase-server.txt
|
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/5675/artifact/patchprocess/patch-unit-hbase-server.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/5675/testReport/ |
| modules | C: hbase-procedure hbase-server U: . |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/5675/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Refactor procedure framework code
> ---------------------------------
>
>                 Key: HBASE-17605
>                 URL: https://issues.apache.org/jira/browse/HBASE-17605
>             Project: HBase
>          Issue Type: Improvement
>          Components: proc-v2
>            Reporter: Appy
>            Assignee: Appy
>         Attachments: HBASE-17605.master.001.patch, HBASE-17605.master.002.patch, HBASE-17605.master.003.patch,
HBASE-17605.master.004.patch, HBASE-17605.master.005.patch, HBASE-17605.master.006.patch,
HBASE-17605.master.007.patch, HBASE-17605.master.008.patch, without-patch.png, with-patch.png
>
>
>     - Moved locks out of MasterProcedureScheduler#Queue. One Queue object is used for
each namespace/table, which aren't more than 100. So we don't complexity arising from all
functionalities being in one place. MasterProcedureLocking#Lock is the new locking class.
>     - Removed NamespaceQueue because it wasn't being used as Queue (add,peek,poll,etc
functions threw UnsupportedOperationException). It's was only used for locks on namespaces.
Now that locks have been moved out of Queue class, it's not needed anymore.
>     - Remoed RegionEvent which was there only for locking on regions. Tables/namespaces
used locking from Queue class and regions couldn't (there are no separate proc queue at region
level), hence the redundance. Now that locking is separate, we can use the same for regions
too.
>     - Removed QueueInterface class. No declarations, except one implementaion, which
makes the point of having an interface moot.
>     - Removed QueueImpl, which was the only concrete implementation of abstract Queue
class. Moved functions to Queue class itself to avoid unnecessary level in inheritance hierarchy.
>     - Removed ProcedureEventQueue class which was just a wrapper around ArrayDeque class.
>     - Encapsulated table priority related stuff in a single class.
>     - Removed some unused functions.
> *Perf using MasterProcedureSchedulerPerformanceEvaluation*
> 10 threads, 10M ops, 5 tables
> Without patch:
> 10 regions/table : #yield 584980, addBack time 4.1s, poll time 10s
> 1M regions/table: #yield 16, addBack time 5.9s, poll time 12.9s
> With patch:
> 10 regions/table : #yield 86413, addBack time 4.1s, poll time 8.2s
> 1M regions/table: #yield 9, addBack time 6s, poll time 13s
> *Memory footprint and CPU* (don't compare GC as that depends on life of objects which
will be much longer in real-world scenarios)
> Without patch
> !without-patch.png|width=800!
> With patch
> !with-patch.png|width=800!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message