hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ning Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2360) create dynamic partition if and only if intermediate source has files
Date Tue, 09 Aug 2011 17:29:27 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081780#comment-13081780
] 

Ning Zhang commented on HIVE-2360:
----------------------------------

Franklin, can you create a review request? That's much easier to comment. Two comments here:


 1) in dynamic partition inserts, a subquery corresponding to a partition is created only
if there is a row that should be inserted into that partition, which means the directory should
not be empty. If the directory is empty, most likely there is a bug there and we should fix
that bug instead. 
 2) the function Utilities.getFileStatusRecurse() is quite expensive. We should avoid using
it and use cached results (it's called somewhere else before). 

> create dynamic partition if and only if intermediate source has files
> ---------------------------------------------------------------------
>
>                 Key: HIVE-2360
>                 URL: https://issues.apache.org/jira/browse/HIVE-2360
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Franklin Hu
>            Assignee: Franklin Hu
>            Priority: Minor
>             Fix For: 0.8.0
>
>         Attachments: hive-2360.1.patch
>
>
> There are some conditions under which a partition description is created due to insert
overwriting a table using dynamic partitioning for partitions that that are empty (have no
files).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message