hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3162) Separate application-init and container-init event types in NM's ApplicationImpl FSM
Date Tue, 11 Oct 2011 06:58:29 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124758#comment-13124758
] 

Vinod Kumar Vavilapalli commented on MAPREDUCE-3162:
----------------------------------------------------

The deletion of app-dirs on each startLocalizer() is only happening in DCE case. And it seems
like a copy-paste from the legacy code from TT. I don't see us deleting app-dirs like that
in LCE case. The corner case mentioned there is also from the MRV1 TT world ("this will happen
on a partial execution of localizeJob."), not sure if it even makes any sense now. Dropping
that delete should do the trick for you at MAPREDUCE-3159?

Regarding the fork, in DCE, we are not even forking, we invoke the localizer directly. Your
comment about avoiding unnecessary fork still applies to LCE though. But we cannot just avoid
the fork directly without other related changes. One problem we've known from before is that
since the app-dirs creation etc are done during the localization itself, not forking when
there are no resources for a container would mean no app-dirs and so containers cannot even
launch! *sigh*

We definitely need to move the creation of app-dirs etc out of resource-localization. That's
a known issue but one which I delayed due to the bigger effort needed.
                
> Separate application-init and container-init event types in NM's ApplicationImpl FSM
> ------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3162
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3162
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Minor
>         Attachments: mr-3162.txt
>
>
> Currently, the ApplicationImpl receives an INIT_APPLICATION event on every container
initialization. Only on the first one does it really mean to init the application, whereas
all subsequent events are for specific containers. This JIRA is to separate the events into
INIT_APPLICATION, sent once and only once per application, and INIT_CONTAINER, which is sent
for every container. The first container sends INIT_APPLICATION followed by INIT_CONTAINER.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message