hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14882) Provide a Put API that adds the provided family, qualifier, value without copying
Date Thu, 01 Dec 2016 13:47:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712007#comment-15712007

Anoop Sam John commented on HBASE-14882:

Thanks for the continuous effort here.  This took much more effort that we initially thought
Some comments
 byte[] buf = new byte[size];
84	    KeyValueUtil.appendToByteArray(this, buf, 0, withTags);
86	    out.write(buf);
This extra copy can be avoided easily.

We add size of 5 refs. All are array type. Means we have to include 5 * ClassSize.ARRAY
There is KeyValue.TIMESTAMP_TYPE_SIZE just use that and make 2 additions into 1.. And this
entire overhead part is going to be constant. Better we can keep in a static final long and
just return that. Avoid calc every time
Suggest create a KV format and return it as a KV object this time. While adding to memstore,
this is been called.  KV is having much lesser heap overhead

Pls do proper formatting of code.. I can see getFamilyOffset is not as per the HBAse guidelines

> Provide a Put API that adds the provided family, qualifier, value without copying
> ---------------------------------------------------------------------------------
>                 Key: HBASE-14882
>                 URL: https://issues.apache.org/jira/browse/HBASE-14882
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 1.2.0
>            Reporter: Jerry He
>            Assignee: Xiang Li
>             Fix For: 2.0.0
>         Attachments: HBASE-14882.master.000.patch, HBASE-14882.master.001.patch, HBASE-14882.master.002.patch,
HBASE-14882.master.003.patch, HBASE-14882.master.004.patch
> In the Put API, we have addImmutable()
> {code}
>  /**
>    * See {@link #addColumn(byte[], byte[], byte[])}. This version expects
>    * that the underlying arrays won't change. It's intended
>    * for usage internal HBase to and for advanced client applications.
>    */
>   public Put addImmutable(byte [] family, byte [] qualifier, byte [] value)
> {code}
> But in the implementation, the family, qualifier and value are still being copied locally
to create kv.
> Hopefully we should provide an API that truly uses immutable family, qualifier and value.

This message was sent by Atlassian JIRA

View raw message