hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "HBase Review Board (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1229) replace dependencies on HBase deprecated API
Date Tue, 06 Jul 2010 21:13:50 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885689#action_12885689
] 

HBase Review Board commented on HIVE-1229:
------------------------------------------

Message from: bkm.hadoop@gmail.com

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/239/#review309
-----------------------------------------------------------



http://svn.apache.org/repos/asf/hadoop/hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
<http://review.hbase.org/r/239/#comment1362>

    I have added a HashMap to HBaseSerDe to cache the String to byte [] mapping. The code
looks it up for the serialization path. The map is also passed to LazyHBaseRow and LazyHBaseCellMap
for lookup during deserialization. In addition I have moved some function calls to serdeParams
and saved their return values as instance variables to reduce the per row calls.



http://svn.apache.org/repos/asf/hadoop/hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSplit.java
<http://review.hbase.org/r/239/#comment1363>

    I have left this in the 3rd patch. Thanks for explaining this. I don't think we have a
test case which exposes this or even whether the serde instance and the record reader instance
could get out of sync, but good idea to leave it in.
    
    The failing tests were due to an improperly initialized Scan instance.



http://svn.apache.org/repos/asf/hadoop/hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat.java
<http://review.hbase.org/r/239/#comment1366>

    This OutputFormat is from the deprecated mapred package - this is needed to keep it compatible
with the storage handler which needs it to be compatible with this -- see the getOutputFormat()
method.



http://svn.apache.org/repos/asf/hadoop/hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseCellMap.java
<http://review.hbase.org/r/239/#comment1365>

    Actually, parse() should also be called only once. I have added the missing call to set
parsed to true. In addition the cached values are now passed in from the serde to lazy row
to the lazy cell map in the deserialization path.



http://svn.apache.org/repos/asf/hadoop/hive/trunk/hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
<http://review.hbase.org/r/239/#comment1364>

    Done, reverted these changes.


- bkm





> replace dependencies on HBase deprecated API
> --------------------------------------------
>
>                 Key: HIVE-1229
>                 URL: https://issues.apache.org/jira/browse/HIVE-1229
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: HBase Handler
>    Affects Versions: 0.6.0
>            Reporter: John Sichi
>            Assignee: Basab Maulik
>             Fix For: 0.7.0
>
>         Attachments: HIVE-1229.1.patch, HIVE-1229.2.patch, HIVE-1229.3.patch
>
>
> Some of these dependencies are on the old Hadoop mapred packages; others are HBase-specific.
 The former have to wait until the rest of Hive moves over to the new Hadoop mapreduce package,
but the HBase-specific ones don't have to wait.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message