hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-15180) Reduce garbage created while reading Cells from Codec Decoder
Date Wed, 27 Jan 2016 12:29:39 GMT
Anoop Sam John created HBASE-15180:

             Summary: Reduce garbage created while reading Cells from Codec Decoder
                 Key: HBASE-15180
                 URL: https://issues.apache.org/jira/browse/HBASE-15180
             Project: HBase
          Issue Type: Sub-task
            Reporter: Anoop Sam John
            Assignee: Anoop Sam John

In KeyValueDecoder#parseCell (Default Codec decoder) we use KeyValueUtil#iscreate to read
cells from the InputStream. Here we 1st create a byte[] of length 4 and read the cell length
and then an array of Cell's length and read in cell bytes into it and create a KV.
Actually in server we read the reqs into a byte[] and CellScanner is created on top of a ByteArrayInputStream
on top of this. By default in write path, we have MSLAB usage ON. So while adding Cells to
memstore, we will copy the Cell bytes to MSLAB memory chunks (default 2 MB size) and recreate
Cells over that bytes.  So there is no issue if we create Cells over the RPC read byte[] directly
here in Decoder.  No need for 2 byte[] creation and copy for every Cell in request.

My plan is to make a Cell aware ByteArrayInputStream which can read Cells directly from it.

Same Codec path is used in client side also. There better we can avoid this direct Cell create
and continue to do the copy to smaller byte[]s path.  Plan to introduce some thing like a
CodecContext associated with every Codec instance which can say the server/client context.

This message was sent by Atlassian JIRA

View raw message