hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15721) Optimization in cloning cells into MSLAB
Date Wed, 05 Oct 2016 17:36:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549445#comment-15549445
] 

Anoop Sam John commented on HBASE-15721:
----------------------------------------

bq.we remove this deprecation which seems to be a regression:
Not really..  The thing is in Cell there can not be any thing like a keybuffer of single buffer.
 But KV, we know is based on single buffer backed.  So a getter for buffer there seems valid
only. KV is private marked also.  We have moved to Cell only flows in read and write paths.
KV is having stuff like getKeyLength, getKeyOffset.  These were never deprecated also..  The
thing was that we were using KV.getBuffer through out our code in 0.96 time.  But now things
are clean.   So IMHO, the getBuffer () in KV is making  sense very much.  If u strongly feel
we should not remove deprecation I can undo that in this patch.. I just came across this API
in this patch and thought there is no point in marking it as deprecated.

Ya we copy the cell into the buf in KV format only. As long as there is no other kind of Cell
impl, we will have to continue with these stuff.. 

> Optimization in cloning cells into MSLAB
> ----------------------------------------
>
>                 Key: HBASE-15721
>                 URL: https://issues.apache.org/jira/browse/HBASE-15721
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15721.patch, HBASE-15721_V2.patch, HBASE-15721_V3.patch, HBASE-15721_V4.patch
>
>
> Before cells added to memstore CSLM, there is a clone of cell after copying it to MSLAB
chunk area.  This is done not in an efficient way.
> {code}
> public static int appendToByteArray(final Cell cell, final byte[] output, final int offset)
{
>     int pos = offset;
>     pos = Bytes.putInt(output, pos, keyLength(cell));
>     pos = Bytes.putInt(output, pos, cell.getValueLength());
>     pos = appendKeyTo(cell, output, pos);
>     pos = CellUtil.copyValueTo(cell, output, pos);
>     if ((cell.getTagsLength() > 0)) {
>       pos = Bytes.putAsShort(output, pos, cell.getTagsLength());
>       pos = CellUtil.copyTagTo(cell, output, pos);
>     }
>     return pos;
>   }
> {code}
> Copied in 9 steps and we end up parsing all lengths.  When the cell implementation is
backed by a single byte[] (Like KeyValue) this can be done in single step copy.
> Also avoid lots of garbage while doing this MSLAB copy. With every cell, we used to create
a ByteRange instance to pass the allocated chunk + offset from MSLAB to Segment.  Now we can
make the MSLAB impl to do the allocation and copy of the cell so that one object creation
per cell addition can be avoided.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message