hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13997) Insert overwrite directory doesn't overwrite existing files
Date Sat, 18 Jun 2016 09:43:02 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15337687#comment-15337687
] 

Rui Li commented on HIVE-13997:
-------------------------------

Hi [~ashutoshc], since we are inserting into directory, we don't need to load table and replaceFiles
won't be called. The call path is {{MoveTask.execute -> moveFile -> moveFileInDfs ->
Hive.moveFile}}. Besides, moveFile does take care of replacing files. I think it just has
a bug when the src is a sub dir of the dest dir.
{{ppd_multi_insert}} fails because {{Hive.isSubDir}} decides whether src is sub dir of dest
by just checking {{src.startsWith(dest)}}. This is incorrect because it returns true when
src is "/a/bc" and dest is "/a/b".
I'll provide a patch to fix this.

> Insert overwrite directory doesn't overwrite existing files
> -----------------------------------------------------------
>
>                 Key: HIVE-13997
>                 URL: https://issues.apache.org/jira/browse/HIVE-13997
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Rui Li
>            Assignee: Rui Li
>         Attachments: HIVE-13997.1.patch
>
>
> Can be easily reproduced by running {{INSERT OVERWRITE DIRECTORY}} to the same dir twice.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message