pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3117) A debug mode in which pig does not delete temporary files
Date Sat, 19 Oct 2013 04:57:42 GMT

    [ https://issues.apache.org/jira/browse/PIG-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13799783#comment-13799783

Cheolsoo Park commented on PIG-3117:

Filed PIG-3528.

> A debug mode in which pig does not delete temporary files
> ---------------------------------------------------------
>                 Key: PIG-3117
>                 URL: https://issues.apache.org/jira/browse/PIG-3117
>             Project: Pig
>          Issue Type: Wish
>    Affects Versions: 0.10.0, 0.11.1
>            Reporter: Ido Hadanny
>            Assignee: Ido Hadanny
>             Fix For: 0.13.0
>         Attachments: PIG-3117-1.patch, PIG-3117-2.patch, remove_intermediate_results.diff
> when we debug our pig jobs on pre-production data, we usually find bugs we couldn't detect
in our UT, as env and data are not quite the same.
> when the final output of a script is not quite what we expect, we start divide-and-conquer,
running it line by line and inspecting the intermediate output of each stage. 
> It would be great if we could simply configure pig not to delete the intermediate MR
outputs, and store them as plaintext instead of snappy format.

This message was sent by Atlassian JIRA

View raw message