hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
Date Thu, 24 Apr 2014 21:31:17 GMT

    [ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13980317#comment-13980317
] 

Sushanth Sowmyan commented on HIVE-6936:
----------------------------------------

Looking into it. Will provide a quick description of how it's handled in HCat:

In HCat, we pass along tableProperties using HiveStorageHandler.configureInputJobProperties
and HiveStorageHandler.configureOutputJobProperties. HBaseSH sees only this.

To standardize usage across all IF/OF cases, we have a storage handler called FosterStorageHandler
that acts as a wrapper around the generic IF/OF cases that do not have storage handlers, so
that the rest of the HCat code can still stick to using StorageHandlers as-is and not worry
about the underlying system.

To make FosterStorageHandler work with orc, we had to change orc so it accepted table property
overrides from jobconf as well (since we have to conform to mapreduce interfaces only, and
that has no knowledge of tables), and had FosterStorageHandler set up jobconf appropriately
with any overrides from table properties.

> Provide table properties to InputFormats
> ----------------------------------------
>
>                 Key: HIVE-6936
>                 URL: https://issues.apache.org/jira/browse/HIVE-6936
>             Project: Hive
>          Issue Type: Bug
>          Components: File Formats
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.14.0
>
>         Attachments: HIVE-6936.patch, HIVE-6936.patch
>
>
> Some advanced file formats need the table properties made available to them. Additionally,
it would be convenient to provide a unique id for fetch operators and the complete list of
directories.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message