hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6749) MR AM should reuse containers for Map/Reduce Tasks
Date Wed, 05 Oct 2016 19:27:20 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549700#comment-15549700

Naganarasimha G R commented on MAPREDUCE-6749:

Thanks [~devaraj.k] for the document and raising these jira's, it has been always discussing
but no progress on this topic lets try to make some progress for this feature here .
Few queries:
# Is it required to have number of total Maps(/reducers) to be run using container reuse?
i felt we can just keep whether to reuse the containers are not and one assumption that map
containers will be reused only for map and reduce only for reduce 
# did not understand the need of *RMContainerReuseRequestor.java*, may be i will go through
MAPREDUCE-6773 but was expecting more like just  TaskAttemptIimpl gets TA_DONE event was expecting
it do have a new ReuseContainerTransition where in if required it will reassign a new task
to it else will continue to *CLEANUP_CONTAINER_TRANSITION*. may be we can discuss more on
this in MAPREDUCE-6773 ?
# If we have some conclusion on issue 2, then we can think of whether only *TaskAttemptImpl*
state is sufficient or we need to modify others too.
# *Logs generated by the tasks in a container will be appended in the same container log file*
we need to ensure we are able to display trhough JHS and other places where ever required
to display

> MR AM should reuse containers for Map/Reduce Tasks
> --------------------------------------------------
>                 Key: MAPREDUCE-6749
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6749
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: applicationmaster, mrv2
>            Reporter: Devaraj K
>            Assignee: Devaraj K
>         Attachments: MAPREDUCE-6749-Container Reuse-v0.pdf
> It is with the continuation of MAPREDUCE-3902, MR AM should reuse containers for Map/Reduce
Tasks similar to the JVM Reuse feature we had in MRv1.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org

View raw message