hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15180) Reduce garbage created while reading Cells from Codec Decoder
Date Thu, 28 Jan 2016 22:59:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15122488#comment-15122488
] 

Enis Soztutar commented on HBASE-15180:
---------------------------------------

This is a good idea. Having all cells in the same RPC share the same byte[]. 

Is {{CellReadable}} really necessary? Isn't this the same thing as Codec.Decoder. I mean,
from a layering perspective, I thought that we would instead change the Codec to be aware
of byte[] directly, and return a CellScanner that can return KV's from the same buffer. I
was thinking of doing a Codec at the RPC layer to do something like FAST_DIFF. Can that still
be done with this patch? 

Should we default to MSLAB for good? I don't think anybody runs with MSLAB off. 

RPCServer reaching this is not right: 
{code}
+    this.mslabEnabled = conf.getBoolean(HConstants.USEMSLAB_KEY, HConstants.USEMSLAB_DEFAULT);
{code}

Can the byte[4]'s be statically allocated?  

> Reduce garbage created while reading Cells from Codec Decoder
> -------------------------------------------------------------
>
>                 Key: HBASE-15180
>                 URL: https://issues.apache.org/jira/browse/HBASE-15180
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15180.patch
>
>
> In KeyValueDecoder#parseCell (Default Codec decoder) we use KeyValueUtil#iscreate to
read cells from the InputStream. Here we 1st create a byte[] of length 4 and read the cell
length and then an array of Cell's length and read in cell bytes into it and create a KV.
> Actually in server we read the reqs into a byte[] and CellScanner is created on top of
a ByteArrayInputStream on top of this. By default in write path, we have MSLAB usage ON. So
while adding Cells to memstore, we will copy the Cell bytes to MSLAB memory chunks (default
2 MB size) and recreate Cells over that bytes.  So there is no issue if we create Cells over
the RPC read byte[] directly here in Decoder.  No need for 2 byte[] creation and copy for
every Cell in request.
> My plan is to make a Cell aware ByteArrayInputStream which can read Cells directly from
it.  
> Same Codec path is used in client side also. There better we can avoid this direct Cell
create and continue to do the copy to smaller byte[]s path.  Plan to introduce some thing
like a CodecContext associated with every Codec instance which can say the server/client context.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message