hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gunther Hagleitner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7220) Empty dir in external table causes issue (root_dir_external_table.q failure)
Date Thu, 19 Jun 2014 20:41:26 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037843#comment-14037843
] 

Gunther Hagleitner commented on HIVE-7220:
------------------------------------------

I think we should move forward with this it will give us a working build, while we work out
MAPREDUCE-5756. We have HIVE-6401 open to handle the situation when we get a fix.

I've reviewed the patch, it looks good except for the isValidSplit call. Why is that needed?
You prune in the constructor so presumably you never get splits containing folders. If this
is just a sanity check it should probably throw an assertion if there's still paths in there.
If not - it seems incorrect to throw out splits that don't match (especially since you might
throw out combined valid locations with it).

> Empty dir in external table causes issue (root_dir_external_table.q failure)
> ----------------------------------------------------------------------------
>
>                 Key: HIVE-7220
>                 URL: https://issues.apache.org/jira/browse/HIVE-7220
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Szehon Ho
>            Assignee: Szehon Ho
>         Attachments: HIVE-7220.patch
>
>
> While looking at root_dir_external_table.q failure, which is doing a query on an external
table located at root ('/'), I noticed that latest Hadoop2 CombineFileInputFormat returns
split representing empty directories (like '/Users'), which leads to failure in Hive's CombineFileRecordReader
as it tries to open the directory for processing.
> Tried with an external table in a normal HDFS directory, and it also returns the same
error.  Looks like a real bug.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message