hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Francis Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-4331) Integrated StorageHandler for Hive and HCat using the HiveStorageHandler
Date Mon, 26 Aug 2013 20:58:52 GMT

    [ https://issues.apache.org/jira/browse/HIVE-4331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13750545#comment-13750545
] 

Francis Liu commented on HIVE-4331:
-----------------------------------

{quote}
For folks who has OF which has OC, it will be easier to integrate that in Hive, instead of
understanding Hive innards and handling of OC. Wondering if you have given a thought on this?
I just want to make sure if and when we go that route these current changes won't get in the
way.
{quote}
For HCat we already do it this way. It's not really just the OC but the OF,OC,RR in general.
HOF essentially is doing the Hive specific stuff that the plain OC, RR, etc can do as well.
So I don't think we changed the complexity of the work needed to support new formats? Is that
what you meant by get in the way? 

In the long run it'd be better since HCat and Hive treat OFs the same way. Though it'd be
great to document what that contract (beyond the typical OF) is. 

                
> Integrated StorageHandler for Hive and HCat using the HiveStorageHandler
> ------------------------------------------------------------------------
>
>                 Key: HIVE-4331
>                 URL: https://issues.apache.org/jira/browse/HIVE-4331
>             Project: Hive
>          Issue Type: Task
>          Components: HCatalog
>    Affects Versions: 0.11.0, 0.12.0
>            Reporter: Ashutosh Chauhan
>            Assignee: Viraj Bhat
>         Attachments: HIVE4331_07-17.patch, StorageHandlerDesign_HIVE4331.pdf
>
>
> 1) Deprecate the HCatHBaseStorageHandler and "RevisionManager" from HCatalog. These will
now continue to function but internally they will use the "DefaultStorageHandler" from Hive.
They will be removed in future release of Hive.
> 2) Design a HivePassThroughFormat so that any new StorageHandler in Hive will bypass
the HiveOutputFormat. We will use this class in Hive's "HBaseStorageHandler" instead of the
"HiveHBaseTableOutputFormat".
> 3) Write new unit tests in the HCat's "storagehandler" so that systems such as Pig and
Map Reduce can use the Hive's "HBaseStorageHandler" instead of the "HCatHBaseStorageHandler".
> 4) Make sure all the old and new unit tests pass without backward compatibility (except
known issues as described in the Design Document).
> 5) Replace all instances of the HCat source code, which point to "HCatStorageHandler"
to use the"HiveStorageHandler" including the "FosterStorageHandler".
> I have attached the design document for the same and will attach a patch to this Jira.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message