hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andy Skelton (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-9389) INSERT OVERWRITE DIRECTORY fails to delete old data files
Date Thu, 15 Jan 2015 17:41:34 GMT
Andy Skelton created HIVE-9389:
----------------------------------

             Summary: INSERT OVERWRITE DIRECTORY fails to delete old data files
                 Key: HIVE-9389
                 URL: https://issues.apache.org/jira/browse/HIVE-9389
             Project: Hive
          Issue Type: Bug
    Affects Versions: 0.13.1
         Environment: CDH 5.3.0, non-secure hdfs, perm checking off
            Reporter: Andy Skelton


{code:sql}
FROM myview INSERT OVERWRITE DIRECTORY 'hdfs://nameservice/path/' SELECT COUNT(DISTINCT mycol);
{code}

This always produces one row. Sometimes the output is two files, {{000000_0}} and {{000000_1}},
one of which is empty. Sometimes we have seen new results in {{000000_0}} while old results
remain in {{000000_1}}.

We were alerted to this because Sqoop was exporting the output files in order by filename,
writing first the new value and then overwriting with the old value, triggering an alert because
the value stopped increasing in our database.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message