hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2668) MR-279: APPLICATION_STOP is never sent to AuxServices
Date Mon, 03 Oct 2011 14:06:36 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119318#comment-13119318

Thomas Graves commented on MAPREDUCE-2668:

some background:

Services are started per container. Theoretically a container could have 0 - X services. The
service can't necessarily STOP when the container is finished - for instance map tasks use
the shuffle handler service to serve up the map output to reduce tasks after mapper is complete.

Right now APPLICATION_STOP isn't sent at all.  Since it can't be sent when container finishes
we need to send it when the application finishes.  The application then either needs to know
what all services the containers used on that particular nm or we broadcast the stop message
to the services and they stop as necessary.  I've chosen to do the second route.

> MR-279: APPLICATION_STOP is never sent to AuxServices
> -----------------------------------------------------
>                 Key: MAPREDUCE-2668
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2668
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Robert Joseph Evans
>            Assignee: Thomas Graves
>            Priority: Blocker
>             Fix For: 0.23.0
> APPLICATION_STOP is never sent to the AuxServices only APPLICATION_INIT.  This means
that all map intermediate data will never be deleted.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message