pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shravan Matthur Narayanamurthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-457) pig produces errors after a job is said to be 100% done
Date Fri, 26 Sep 2008 19:57:44 GMT

    [ https://issues.apache.org/jira/browse/PIG-457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634984#action_12634984

Shravan Matthur Narayanamurthy commented on PIG-457:

The patch also removes the fix which resolves 2 where it shows hundred percent even after
failures. Also, pls call jc.stop before returning false from the failure case.  Also, the
LocalJobRunner.compile has the StreamVisitor thingy missing which I think causes the stream
tests in local mode to fail. If thats actually the case, i would keep that in too.

Optionally we might want to keep the check that discards progress which is decreasing because
it causes the condition we have there of prog<lastProg to fail. The progress is like x
and lastProg is also x. This goes on for a while and suddenly, probably during failure, the
prog becomes zero which causes the condition to hold in the next iteration and makes us print
the same progress again and again.

The exponential backoff needs some review I agree

> pig produces errors after a job is said to be 100% done
> -------------------------------------------------------
>                 Key: PIG-457
>                 URL: https://issues.apache.org/jira/browse/PIG-457
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Olga Natkovich
>            Assignee: Shravan Matthur Narayanamurthy
>             Fix For: types_branch
>         Attachments: 457-2.patch, 457-3.patch
> It is possible that we get errors for all tasks even the ones we retried. Need to look
at the code that handles detecting end of processing and producing errors.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message