hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carl Steinbach (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-2126) Hive's symlink text input format should be able to work with ComineHiveInputFormat
Date Tue, 05 Jul 2011 02:37:21 GMT

     [ https://issues.apache.org/jira/browse/HIVE-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Carl Steinbach updated HIVE-2126:
---------------------------------

    Fix Version/s: 0.8.0

> Hive's symlink text input format should be able to work with ComineHiveInputFormat
> ----------------------------------------------------------------------------------
>
>                 Key: HIVE-2126
>                 URL: https://issues.apache.org/jira/browse/HIVE-2126
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: He Yongqiang
>            Assignee: He Yongqiang
>             Fix For: 0.8.0
>
>         Attachments: HIVE-2126.1.patch, HIVE-2126.2.patch
>
>
> at compile time, if a partition's file format is SymlinkTextInputFormat, will replace
the symlink path with paths in the symlink file. This way, it will work with Hive's HiveCombineFileInputFormat.
> The reason we are doing it at compile time is because:
> 1) At run time, the input path is not only used to get record reader, but also used for
hive to get aliases and thus operator tree. But the CombineHiveInputFormat can have multiple
paths for each split, and when switching paths, it also set the job with new input file name.
So it always require a real input path name. Can not fake it.
> 2) if write a new input format, it will require a lot of duplication work with existing
CombineHiveInputFormat.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message