hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2756) JobControl can drop jobs if an error occurs
Date Tue, 16 Aug 2011 17:57:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085872#comment-13085872
] 

Robert Joseph Evans commented on MAPREDUCE-2756:
------------------------------------------------

I did not provide a patch for trunk because this mostly impacts MR-279.  It is rather difficult
to cause an Error instead of an Exception to happen except with the change in binary compatibility
caused by MR-279.  It think it would be simpler to just wait for MR-279 to merge to trunk
before applying the patch.

As far as manual testing.  We ran about 100 different pig scripts with custom Storage functions
in them.  Before the change one of them would consistently deadlock, and all the others would
fail, but with no indication of why.  After the change they all still fail.  No deadlocks,
and there is a log message indicating why it failed. 

> JobControl can drop jobs if an error occurs
> -------------------------------------------
>
>                 Key: MAPREDUCE-2756
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2756
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: client, mrv2
>    Affects Versions: 0.23.0
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>            Priority: Minor
>             Fix For: 0.23.0
>
>         Attachments: MR-2756-V1.patch
>
>
> If you run a pig job with UDFs that has not been recompiled for MRV2.  There are situations
where pig will fail with an error message stating that Hadoop failed and did not give a reason.
 There is even the possibility of deadlock if an Error is thrown and the JobControl thread
dies.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message