hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-258) Pig should cleanup output directory of a failed query
Date Wed, 02 Jul 2008 00:18:46 GMT

    [ https://issues.apache.org/jira/browse/PIG-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12609752#action_12609752
] 

Olga Natkovich commented on PIG-258:
------------------------------------

Hi Daniel,

Looks good. I have a couple of questions/comments:

(1) I don't think we should through exception if we can't register the file to delete. We
should just log a warning
(2) Also, if we can't delete, we should log a warning not an error
(3) I think you might be double deleting in case of dump. dump stores into temp file which
would get cleaned up when we remove temp files. This might be ok if both check for file existance
before trying to delete
(3) What kind of tests did you run?

I think we need to test the following:

(1) Pig script with a single successful store
(2) Pig script with multiple successful stores
(3) Pig script with one failed store 
(4) Pig script with multiple failures 
(5) Pig script with one successful and one failed store
(6) Pig script with a single successful dump
(7) Pig script with a single failed dump



> Pig should cleanup output directory of a failed query
> -----------------------------------------------------
>
>                 Key: PIG-258
>                 URL: https://issues.apache.org/jira/browse/PIG-258
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Olga Natkovich
>            Priority: Minor
>         Attachments: clearoutput.patch
>
>
> Currently, after a failed store, the output directory is left behind and can't be re-used
without manual cleanup

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message