hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Sichi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1083) allow sub-directories for an external table/partition
Date Sat, 24 Apr 2010 00:57:50 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860450#action_12860450

John Sichi commented on HIVE-1083:

Clarification:  you can already get the desired behavior using HDFS, MAPREDUCE-1501, and mapred.input.dir.recursive=true,
as long as your query doesn't hit one of the corner cases enuemrated by Zheng.

What remains for this task are the following:

(1) support local file system as well (this failed when I tested it, but I didn't look into

(2) deal with HIVE-1133 to fix the corner cases

> allow sub-directories for an external table/partition
> -----------------------------------------------------
>                 Key: HIVE-1083
>                 URL: https://issues.apache.org/jira/browse/HIVE-1083
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Query Processor
>    Affects Versions: 0.6.0
>            Reporter: Namit Jain
>            Assignee: Zheng Shao
>             Fix For: 0.6.0
> Sometimes users want to define an external table/partition based on all files (recursively)
inside a directory.
> Currently most of the Hadoop InputFormat classes do not support that. We should extract
all files recursively in the directory, and add them to the input path of the job.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message