hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheng Hao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-3823) Performance issue while retrieving the Result objects in HiveHBaseTableInputFormat
Date Thu, 20 Dec 2012 01:13:12 GMT

     [ https://issues.apache.org/jira/browse/HIVE-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Cheng Hao updated HIVE-3823:
----------------------------

    Description: 
In HiveHBaseTableInputFormat.java, the Result objects retrieving has  performance issue.
{code:title=HiveHBaseTableInputFormat.java}
      @Override
      public boolean next(ImmutableBytesWritable rowKey, Result value) throws IOException
{

        boolean next = false;

        try {
          next = recordReader.nextKeyValue();

          if (next) {
            rowKey.set(recordReader.getCurrentValue().getRow());
            // performance issue here, as the copyWritable
            // is Serialization - Bytes Copying - Deserialization.
            Writables.copyWritable(recordReader.getCurrentValue(), value);
          }
        } catch (InterruptedException e) {
          throw new IOException(e);
        }

        return next;
      }
{code}

In HBASE 0.94.4 & 0.96.0, the Result provides a new method "copyFrom(Result from)", will
solve the issue.

See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]


  was:
In HiveHBaseTableInputFormat.java, the Result objects retrieving has  performance issue.
{code:title=HiveHBaseTableInputFormat.java}
      @Override
      public boolean next(ImmutableBytesWritable rowKey, Result value) throws IOException
{

        boolean next = false;

        try {
          next = recordReader.nextKeyValue();

          if (next) {
            rowKey.set(recordReader.getCurrentValue().getRow());
            // performance issue here, as the copyWritable
            // is Serialization - Bytes Copying - Deserialization.
            Writables.copyWritable(recordReader.getCurrentValue(), value);
          }
        } catch (InterruptedException e) {
          throw new IOException(e);
        }

        return next;
      }
{code}

In HBASE 0.94.4 & 0.96.0, the Result provides a new method copyFrom(Result from), would
solve the issue.

See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]


    
> Performance issue while retrieving the Result objects in HiveHBaseTableInputFormat
> ----------------------------------------------------------------------------------
>
>                 Key: HIVE-3823
>                 URL: https://issues.apache.org/jira/browse/HIVE-3823
>             Project: Hive
>          Issue Type: Improvement
>          Components: HBase Handler
>    Affects Versions: 0.9.0, 0.10.0, 0.9.1
>            Reporter: Cheng Hao
>            Priority: Trivial
>
> In HiveHBaseTableInputFormat.java, the Result objects retrieving has  performance issue.
> {code:title=HiveHBaseTableInputFormat.java}
>       @Override
>       public boolean next(ImmutableBytesWritable rowKey, Result value) throws IOException
{
>         boolean next = false;
>         try {
>           next = recordReader.nextKeyValue();
>           if (next) {
>             rowKey.set(recordReader.getCurrentValue().getRow());
>             // performance issue here, as the copyWritable
>             // is Serialization - Bytes Copying - Deserialization.
>             Writables.copyWritable(recordReader.getCurrentValue(), value);
>           }
>         } catch (InterruptedException e) {
>           throw new IOException(e);
>         }
>         return next;
>       }
> {code}
> In HBASE 0.94.4 & 0.96.0, the Result provides a new method "copyFrom(Result from)",
will solve the issue.
> See [HBASE-7381|https://issues.apache.org/jira/browse/HBASE-7381]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message