hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-9535) Try a pool of direct byte buffers handling incoming ipc requests
Date Sun, 15 Sep 2013 04:30:53 GMT
stack created HBASE-9535:

             Summary: Try a pool of direct byte buffers handling incoming ipc requests
                 Key: HBASE-9535
                 URL: https://issues.apache.org/jira/browse/HBASE-9535
             Project: HBase
          Issue Type: Brainstorming
            Reporter: stack
            Assignee: stack

ipc takes in a query by allocating a ByteBuffer of the size of the request and then reading
off the socket into this on-heap BB.

Experiment with keeping a pool of BBs so we have some buffer reuse to cut on garbage generated.
 Could checkout from pool in RpcServer#Reader.  Could check back into the pool when Handler
is done just before it queues the response on the Responder's queue.  We should be good since,
at least for now, kvs get copied up into MSLAB (not references) when data gets stuffed into
MemStore; this should make it so no references left over when we check the BB back into the
pool for use next time around.

If on-heap BBs work, we could then try direct BBs (Allocation of DBBs takes time so if already
allocated, should be good.  GC of DBBs is a pain but if in a pool, we shouldn't be wanting
this to happen).  The copy from socket to the DBB will be off-heap (should be fast).

Could start w/ the HDFS DirectBufferPool.  It is unbounded and keeps items by size (we might
want to bypass the pool if an object is > size N).

DBBs for this task would contend w/ offheap BBs used in BlockReadLocal when short-circuit
reading.  It'd be a bummer if we had to allocate big objects on-heap.  Would still be an improvement.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message