hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shravan Matthur Narayanamurthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-457) pig produces errors after a job is said to be 100% done
Date Thu, 02 Oct 2008 01:04:44 GMT

    [ https://issues.apache.org/jira/browse/PIG-457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636261#action_12636261
] 

Shravan Matthur Narayanamurthy commented on PIG-457:
----------------------------------------------------

Actually, the final progress check is exactly that. It doesn't report 100% until it is also
sure that the job completed successfully. However, there are 2 other changes: 1) to mask some
issue with hadoop progress 2) When the issue (1) is masked we would not see any kind of progress
if the job is not showing progress after a certain point of time. So instead appearing frozen,
there is some logic which displays the same percentage with an exponentially increasing delay
between each report of the same progress just to say that we are alive. Once we get a different
progress all counters are reset and it resumes normal progress reporting until the progress
is blocked again. (Not exactly right but just to simplify understanding. It actually adds
exponential delay between a bunch of same progress reports).

> pig produces errors after a job is said to be 100% done
> -------------------------------------------------------
>
>                 Key: PIG-457
>                 URL: https://issues.apache.org/jira/browse/PIG-457
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Olga Natkovich
>            Assignee: Shravan Matthur Narayanamurthy
>             Fix For: types_branch
>
>         Attachments: 457-2.patch, 457-3.patch, 457-4.patch, 457-part2.patch
>
>
> It is possible that we get errors for all tasks even the ones we retried. Need to look
at the code that handles detecting end of processing and producing errors.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message