Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D416C18520 for ; Thu, 28 Jan 2016 05:31:52 +0000 (UTC) Received: (qmail 40943 invoked by uid 500); 28 Jan 2016 05:31:40 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 40901 invoked by uid 500); 28 Jan 2016 05:31:40 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 40887 invoked by uid 99); 28 Jan 2016 05:31:40 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Jan 2016 05:31:40 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id D89702C1F54 for ; Thu, 28 Jan 2016 05:31:39 +0000 (UTC) Date: Thu, 28 Jan 2016 05:31:39 +0000 (UTC) From: "Hadoop QA (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-15180) Reduce garbage created while reading Cells from Codec Decoder MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15120788#comment-15120788 ] Hadoop QA commented on HBASE-15180: ----------------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 3 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 10s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s {color} | {color:green} master passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 17s {color} | {color:green} master passed with JDK v1.7.0_91 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m 56s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 42s {color} | {color:green} master passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 50s {color} | {color:red} hbase-common in master has 1 extant Findbugs warnings. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 6s {color} | {color:red} hbase-server in master has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s {color} | {color:green} master passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 18s {color} | {color:green} master passed with JDK v1.7.0_91 {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 16s {color} | {color:red} hbase-client in the patch failed. {color} | | {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 29s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 26s {color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_66. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 26s {color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_66. {color} | | {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 29s {color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_91. {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 29s {color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_91. {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 11s {color} | {color:red} Patch generated 2 new checkstyle issues in hbase-common (total was 100, now 102). {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 40s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 25m 35s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 18s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s {color} | {color:green} the patch passed with JDK v1.8.0_66 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s {color} | {color:green} the patch passed with JDK v1.7.0_91 {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 52s {color} | {color:green} hbase-client in the patch passed with JDK v1.8.0_66. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 43s {color} | {color:green} hbase-common in the patch passed with JDK v1.8.0_66. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 29s {color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_66. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 59s {color} | {color:green} hbase-client in the patch passed with JDK v1.7.0_91. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 47s {color} | {color:green} hbase-common in the patch passed with JDK v1.7.0_91. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 27s {color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_91. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 25s {color} | {color:green} Patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 72m 36s {color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.9.1 Server=1.9.1 Image:yetus/hbase:date2016-01-28 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12784814/HBASE-15180.patch | | JIRA Issue | HBASE-15180 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux d49b4bbb6a97 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / 47c4147 | | findbugs | v3.0.0 | | findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/324/artifact/patchprocess/branch-findbugs-hbase-common-warnings.html | | findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/324/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html | | mvninstall | https://builds.apache.org/job/PreCommit-HBASE-Build/324/artifact/patchprocess/patch-mvninstall-hbase-client.txt | | mvninstall | https://builds.apache.org/job/PreCommit-HBASE-Build/324/artifact/patchprocess/patch-mvninstall-hbase-server.txt | | compile | https://builds.apache.org/job/PreCommit-HBASE-Build/324/artifact/patchprocess/patch-compile-hbase-server-jdk1.8.0_66.txt | | javac | https://builds.apache.org/job/PreCommit-HBASE-Build/324/artifact/patchprocess/patch-compile-hbase-server-jdk1.8.0_66.txt | | compile | https://builds.apache.org/job/PreCommit-HBASE-Build/324/artifact/patchprocess/patch-compile-hbase-server-jdk1.7.0_91.txt | | javac | https://builds.apache.org/job/PreCommit-HBASE-Build/324/artifact/patchprocess/patch-compile-hbase-server-jdk1.7.0_91.txt | | checkstyle | https://builds.apache.org/job/PreCommit-HBASE-Build/324/artifact/patchprocess/diff-checkstyle-hbase-common.txt | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/324/artifact/patchprocess/patch-unit-hbase-server-jdk1.8.0_66.txt | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/324/artifact/patchprocess/patch-unit-hbase-server-jdk1.7.0_91.txt | | JDK v1.7.0_91 Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/324/testReport/ | | modules | C: hbase-client hbase-common hbase-server U: . | | Max memory used | 174MB | | Powered by | Apache Yetus 0.1.0 http://yetus.apache.org | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/324/console | This message was automatically generated. > Reduce garbage created while reading Cells from Codec Decoder > ------------------------------------------------------------- > > Key: HBASE-15180 > URL: https://issues.apache.org/jira/browse/HBASE-15180 > Project: HBase > Issue Type: Sub-task > Components: regionserver > Reporter: Anoop Sam John > Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-15180.patch > > > In KeyValueDecoder#parseCell (Default Codec decoder) we use KeyValueUtil#iscreate to read cells from the InputStream. Here we 1st create a byte[] of length 4 and read the cell length and then an array of Cell's length and read in cell bytes into it and create a KV. > Actually in server we read the reqs into a byte[] and CellScanner is created on top of a ByteArrayInputStream on top of this. By default in write path, we have MSLAB usage ON. So while adding Cells to memstore, we will copy the Cell bytes to MSLAB memory chunks (default 2 MB size) and recreate Cells over that bytes. So there is no issue if we create Cells over the RPC read byte[] directly here in Decoder. No need for 2 byte[] creation and copy for every Cell in request. > My plan is to make a Cell aware ByteArrayInputStream which can read Cells directly from it. > Same Codec path is used in client side also. There better we can avoid this direct Cell create and continue to do the copy to smaller byte[]s path. Plan to introduce some thing like a CodecContext associated with every Codec instance which can say the server/client context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)