hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheng Hao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7381) Lightweight data transfer for Class Result
Date Tue, 18 Dec 2012 05:56:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534679#comment-13534679
] 

Cheng Hao commented on HBASE-7381:
----------------------------------

@Yu
    The following code is from the Hive HBaseHandler, you may need to notice the part I commented
out.

{code:title=HiveHBaseTableInputFormat.java|borderStyle=solid}
      @Override
      public boolean next(ImmutableBytesWritable rowKey, Result value) throws IOException
{

        boolean next = false;

        try {
          next = recordReader.nextKeyValue();

          if (next) {
            rowKey.set(recordReader.getCurrentValue().getRow());
            Writables.copyWritable(recordReader.getCurrentValue(), value);
            //Result.copy(recordReader.getCurrentValue(), value);
          }
        } catch (InterruptedException e) {
          throw new IOException(e);
        }

        return next;
      }
{code} 

                
> Lightweight data transfer for Class Result
> ------------------------------------------
>
>                 Key: HBASE-7381
>                 URL: https://issues.apache.org/jira/browse/HBASE-7381
>             Project: HBase
>          Issue Type: Improvement
>          Components: Client
>            Reporter: Cheng Hao
>            Priority: Trivial
>             Fix For: 0.94.4
>
>         Attachments: result_lightweight_copy.patch
>
>
> Currently,the data transferring between 2 Result objects in the same process, will cause
additional/unnecessary data parsing & copying; as we have to do that via "Writables.copyWritable(result1,
result2)", which internally is serialization, data copying, and de-serialization.
> The use case are quite common when integrated with Hadoop job running;
> The protocol org.apache.hadoop.mapred.RecordReader defined in Hadoop, provides 3 interfaces:
> 1) K createKey();
> 2) V createValue();
> 3) boolean next(K key, V value) throws IOException;
> In the 3rd method implementation, most likely requires the value (should be Result object)
to be filled, with the Result object from HBase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message