hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3484) Job end notification method should be called before stop() in handle(JobFinishEvent)
Date Wed, 14 Dec 2011 23:49:31 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169819#comment-13169819
] 

Vinod Kumar Vavilapalli commented on MAPREDUCE-3484:
----------------------------------------------------

Patch looks good to me. +1.

I was concerned about use cases like oozie which act on the notification url and request JobCounters
on notification, but I think we are fine because
 - if the notification is received by oozie and the AM is still alive, AM will directly respond
to the request for counters
 - if the notification is received but the AM is gone by then, it'd mean that history is properly
written and JobHistoryServer can respond to the request.

There is a small corner case when AM sends the notification but crashes before completely
writing the JobHistory. In this case, the client/oozie will see the job as successful via
RPC/UI, but cannot get any information. This is an orthogonal bug, I'll create a separate
ticket.

At any rate, I think we are missing an integration test for job-end notification feature.
We need to modify/port {{NotificationTestCase}} to work with {{MiniMRYarnCluster}}. Will open
a ticket for that too.
                
> Job end notification method should be called before stop() in handle(JobFinishEvent)
> ------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3484
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3484
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mr-am, mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ravi Prakash
>            Assignee: Ravi Prakash
>         Attachments: MAPREDUCE-3484.branch-0.23.patch, MAPREDUCE-3484.branch-0.23.patch,
MAPREDUCE-3484.branch-0.23.patch
>
>
> We noticed JobEndNotifier was getting an InterruptedException before completing all its
retries.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message