hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-19279) remove magic directory skipping from CopyTask
Date Mon, 23 Apr 2018 21:24:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-19279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sergey Shelukhin updated HIVE-19279:
------------------------------------
    Description: 
Follow up from HIVE-17657.
Code exists in copytask that copies files (fancy that); however, when listing the files, if
a single directory exists at the source with no other files, it will skip the directory and
copy the files inside instead.
This directory in various tests is either the "data" directory from export, or some random
partition directory ("foo=bar") that if not skipped makes it into the real partition directory
at the destination.
It won't do that if any other files or directories are present.

This seems brittle. Caller of the CopyTask should specify exactly what it wants copied instead
of relying on this behavior.

  was:
Follow up from HIVE-17657.
Code exists in copytask that copies files (fancy that); however, when listing the files, if
a single directory exists at the source with no other files, it will skip the directory and
copy the files inside instead.
This directory in various tests happens to be the "data" directory from export, or some random
partition directory ("foo=bar") that if not skipped makes it into the real partition directory
at the destination.
It won't do that if any other files or directories are present.

This seems brittle. Caller of the CopyTask should specify exactly what it wants copied instead
of relying on this behavior.


> remove magic directory skipping from CopyTask
> ---------------------------------------------
>
>                 Key: HIVE-19279
>                 URL: https://issues.apache.org/jira/browse/HIVE-19279
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Priority: Major
>
> Follow up from HIVE-17657.
> Code exists in copytask that copies files (fancy that); however, when listing the files,
if a single directory exists at the source with no other files, it will skip the directory
and copy the files inside instead.
> This directory in various tests is either the "data" directory from export, or some random
partition directory ("foo=bar") that if not skipped makes it into the real partition directory
at the destination.
> It won't do that if any other files or directories are present.
> This seems brittle. Caller of the CopyTask should specify exactly what it wants copied
instead of relying on this behavior.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message