hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Sichi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1083) allow sub-directories for an external table/partition
Date Sat, 24 Apr 2010 00:57:50 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860450#action_12860450
] 

John Sichi commented on HIVE-1083:
----------------------------------

Clarification:  you can already get the desired behavior using HDFS, MAPREDUCE-1501, and mapred.input.dir.recursive=true,
as long as your query doesn't hit one of the corner cases enuemrated by Zheng.

What remains for this task are the following:

(1) support local file system as well (this failed when I tested it, but I didn't look into
why)

(2) deal with HIVE-1133 to fix the corner cases




> allow sub-directories for an external table/partition
> -----------------------------------------------------
>
>                 Key: HIVE-1083
>                 URL: https://issues.apache.org/jira/browse/HIVE-1083
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Query Processor
>    Affects Versions: 0.6.0
>            Reporter: Namit Jain
>            Assignee: Zheng Shao
>             Fix For: 0.6.0
>
>
> Sometimes users want to define an external table/partition based on all files (recursively)
inside a directory.
> Currently most of the Hadoop InputFormat classes do not support that. We should extract
all files recursively in the directory, and add them to the input path of the job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message