hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9535) Try a pool of direct byte buffers handling incoming ipc requests
Date Fri, 29 Nov 2013 11:42:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13835330#comment-13835330
] 

Hadoop QA commented on HBASE-9535:
----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12616360/9535.v2-trunk.patch
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 1.0 profile.

    {color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 2.0 profile.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 findbugs{color}.  The patch appears to introduce 1 new Findbugs (version
1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

    {color:red}-1 site{color}.  The patch appears to cause mvn site goal to fail.

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.ipc.TestIPC

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8029//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8029//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8029//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8029//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8029//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8029//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8029//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8029//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8029//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8029//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8029//console

This message is automatically generated.

> Try a pool of direct byte buffers handling incoming ipc requests
> ----------------------------------------------------------------
>
>                 Key: HBASE-9535
>                 URL: https://issues.apache.org/jira/browse/HBASE-9535
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: stack
>            Assignee: Nicolas Liochon
>         Attachments: 9535.v1.patch, 9535.v2-trunk.patch, 9535.v2.patch, 9535.v3.patch
>
>
> ipc takes in a query by allocating a ByteBuffer of the size of the request and then reading
off the socket into this on-heap BB.
> Experiment with keeping a pool of BBs so we have some buffer reuse to cut on garbage
generated.  Could checkout from pool in RpcServer#Reader.  Could check back into the pool
when Handler is done just before it queues the response on the Responder's queue.  We should
be good since, at least for now, kvs get copied up into MSLAB (not references) when data gets
stuffed into MemStore; this should make it so no references left over when we check the BB
back into the pool for use next time around.
> If on-heap BBs work, we could then try direct BBs (Allocation of DBBs takes time so if
already allocated, should be good.  GC of DBBs is a pain but if in a pool, we shouldn't be
wanting this to happen).  The copy from socket to the DBB will be off-heap (should be fast).
> Could start w/ the HDFS DirectBufferPool.  It is unbounded and keeps items by size (we
might want to bypass the pool if an object is > size N).
> DBBs for this task would contend w/ offheap BBs used in BlockReadLocal when short-circuit
reading.  It'd be a bummer if we had to allocate big objects on-heap.  Would still be an improvement.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message