hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <>
Subject [jira] [Commented] (HIVE-4331) Integrated StorageHandler for Hive and HCat using the HiveStorageHandler
Date Mon, 26 Aug 2013 18:49:52 GMT


Sushanth Sowmyan commented on HIVE-4331:

I would love to see SH become a first class entry in hive, and HOF be a kind of SH, leading
to its eventual removal. That's precisely what my long-term goal for this is.

> and I'm not completely certain people have a need for that

By this bit, I meant that I wasn't sure people had a need for doing away with HOF (other than
for code-cleanliness, which is why I would like to see it gone) being able to use any generic
OF with hive - most of the usecases of traditional M/R OFs are already covered by hive, or
for newer formats being developed, the OF writer winds up making changes so that it is hive
compatible, such as with orc, or with the HBase SH. So unless there were a major push to see
a BlahOutputFormat that is widely used, but was not already usable from within Hive, I don't
see there being a necessity case for a change in hive that I want. :)
> Integrated StorageHandler for Hive and HCat using the HiveStorageHandler
> ------------------------------------------------------------------------
>                 Key: HIVE-4331
>                 URL:
>             Project: Hive
>          Issue Type: Task
>          Components: HCatalog
>    Affects Versions: 0.11.0, 0.12.0
>            Reporter: Ashutosh Chauhan
>            Assignee: Viraj Bhat
>         Attachments: HIVE4331_07-17.patch, StorageHandlerDesign_HIVE4331.pdf
> 1) Deprecate the HCatHBaseStorageHandler and "RevisionManager" from HCatalog. These will
now continue to function but internally they will use the "DefaultStorageHandler" from Hive.
They will be removed in future release of Hive.
> 2) Design a HivePassThroughFormat so that any new StorageHandler in Hive will bypass
the HiveOutputFormat. We will use this class in Hive's "HBaseStorageHandler" instead of the
> 3) Write new unit tests in the HCat's "storagehandler" so that systems such as Pig and
Map Reduce can use the Hive's "HBaseStorageHandler" instead of the "HCatHBaseStorageHandler".
> 4) Make sure all the old and new unit tests pass without backward compatibility (except
known issues as described in the Design Document).
> 5) Replace all instances of the HCat source code, which point to "HCatStorageHandler"
to use the"HiveStorageHandler" including the "FosterStorageHandler".
> I have attached the design document for the same and will attach a patch to this Jira.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message