hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15177) Reduce garbage created under high load
Date Tue, 02 Feb 2016 21:40:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129088#comment-15129088
] 

Hadoop QA commented on HBASE-15177:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue}
Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} |
{color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color}
| {color:green} The patch appears to include 1 new or modified test files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 59s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 25s {color} |
{color:green} master passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 19s {color} |
{color:green} master passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 8m 5s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 43s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 21s {color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s {color} |
{color:green} master passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 32s {color} |
{color:green} master passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 17s {color} | {color:red}
hbase-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 31s {color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 27s {color} | {color:red}
hbase-server in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 27s {color} | {color:red}
hbase-server in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 29s {color} | {color:red}
hbase-server in the patch failed with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 29s {color} | {color:red}
hbase-server in the patch failed with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 19s {color} | {color:red}
Patch generated 3 new checkstyle issues in hbase-common (total was 7, now 10). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 36s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color}
| {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 22m 18s {color}
| {color:green} Patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2
2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 55s {color} | {color:red}
hbase-common introduced 1 new FindBugs issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s {color} | {color:green}
the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 12s {color} |
{color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 48s {color} | {color:red}
hbase-client in the patch failed with JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 36s {color} | {color:green}
hbase-common in the patch passed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 20s {color} | {color:red}
hbase-server in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 53s {color} | {color:red}
hbase-client in the patch failed with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 42s {color} | {color:green}
hbase-common in the patch passed with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 25s {color} | {color:red}
hbase-server in the patch failed with JDK v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 23s {color}
| {color:green} Patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 11s {color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-common |
|  |  org.apache.hadoop.hbase.util.Threads.LOGGING_EXCEPTION_HANDLER isn't final but should
be  At Threads.java:be  At Threads.java:[line 48] |
| JDK v1.8.0_66 Failed junit tests | hadoop.hbase.client.TestClientScanner |
| JDK v1.7.0_91 Failed junit tests | hadoop.hbase.client.TestClientScanner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.9.1 Server=1.9.1 Image:yetus/hbase:date2016-02-02 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12785843/hbase-15177_v2.patch
|
| JIRA Issue | HBASE-15177 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  hbaseanti  checkstyle
 compile  |
| uname | Linux 065e7d1e58e4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / ed46591 |
| findbugs | v3.0.0 |
| mvninstall | https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/patch-mvninstall-hbase-client.txt
|
| mvninstall | https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/patch-mvninstall-hbase-server.txt
|
| compile | https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/patch-compile-hbase-server-jdk1.8.0_66.txt
|
| javac | https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/patch-compile-hbase-server-jdk1.8.0_66.txt
|
| compile | https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/patch-compile-hbase-server-jdk1.7.0_91.txt
|
| javac | https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/patch-compile-hbase-server-jdk1.7.0_91.txt
|
| checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/diff-checkstyle-hbase-common.txt
|
| findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/new-findbugs-hbase-common.html
|
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/patch-unit-hbase-client-jdk1.8.0_66.txt
|
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/patch-unit-hbase-server-jdk1.8.0_66.txt
|
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/patch-unit-hbase-client-jdk1.7.0_91.txt
|
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/patch-unit-hbase-server-jdk1.7.0_91.txt
|
| unit test logs |  https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/patch-unit-hbase-client-jdk1.8.0_66.txt
https://builds.apache.org/job/PreCommit-HBASE-Build/402/artifact/patchprocess/patch-unit-hbase-client-jdk1.7.0_91.txt
|
| JDK v1.7.0_91  Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/402/testReport/
|
| modules | C: hbase-client hbase-common hbase-server U: . |
| Max memory used | 174MB |
| Powered by | Apache Yetus 0.1.0   http://yetus.apache.org |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/402/console |


This message was automatically generated.



> Reduce garbage created under high load
> --------------------------------------
>
>                 Key: HBASE-15177
>                 URL: https://issues.apache.org/jira/browse/HBASE-15177
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>             Fix For: 2.0.0, 1.3.0
>
>         Attachments: Screen Shot 2016-01-26 at 10.03.48 PM.png, Screen Shot 2016-01-26
at 10.03.56 PM.png, Screen Shot 2016-01-26 at 10.06.16 PM.png, Screen Shot 2016-01-26 at 10.15.15
PM.png, hbase-15177_v0.patch, hbase-15177_v1.patch, hbase-15177_v2.patch
>
>
> I have been doing some profiling of the garbage being created. The idea was to follow
up on HBASE-14490 and experiment with offheap IPC byte buffers and byte buffer re-use. However,
without changing the IPC byte buffers for now, there are a couple of (easy) improvements that
I've identified from profiling: 
> 1. RPCServer.Connection.processRequest() should work with ByteBuffer instead of byte[]
and not-recreate CodedInputStream a few times. 
> 2. RSRpcServices.getRegion() allocates two byte arrays for region, while only 1 is needed.
> 3. AnnotationReadingPriorityFunction is very expensive in allocations. Mainly it allocates
the regionName byte[] to get the table name. We already set the priority for most of the operations
(multi, get, increment, etc) but we are only reading the priority in case of multi. We should
use the priority from the client side. 
> Lets do the simple improvements in this patch, we can get to IPC buffer re-use in HBASE-14490.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message