hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj K (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6773) Implement RM Container Reuse Requestor to handle the reuse containers for resource requests
Date Fri, 28 Oct 2016 00:53:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15613847#comment-15613847

Devaraj K commented on MAPREDUCE-6773:

Thanks [~Naganarasimha] for the comments and sorry for the delay.

I am working on this to update the patch, before providing patch I would like to confirm these.

bq. RMContainerReUseRequestor which specifically deals with the handling of structures required
for the requesting, but any other better name as it not exactly requesting the container but
just storing the stats required for it ?
Do you have any suggestion for this name or anything in your mind which you think better?

bq. IIUC, Though superficially container.reuse.max-maptaskcontainers looks usefull, again
was wondering how the job submitter/admin is going to configure this max # of concurrent MAP(/reduce)
container which can be reused ? atleast the previous one(container.reuse.max-maptasks) based
on the historic values, admin can determine how much time one map/reduce task is going to
take time and accordingly ask the job submitter to configure but this concurrent max reuse
(MAP/REDUCE) tasks is just a blind configuration and would be difficult to determine the right
value. Thoughts ? (hope my understanding of this config is correct !).
Here we are providing this configuration to leave the headroom for other applications/jobs
instead of using all the resources for reuse containers of one Job(if there is a need of resources
for the Job). The User/admin would be having idea of what workloads going to run, they can
decide to configure value for this based on the coming jobs while running the current Job.

bq. may be we can split this jira into 2 parts like creating the interface for ContainerReqestor
and either in new jira or in MAPREDUCE-6786, add the code related to reuse so that its complete.
as these jiras doesnt look to be testable pieces of jiras.
You mean, we need to move the ContainerReqestor interface definition to another jira or the
instantiation of the ContainerReqestor?

bq. has white space errors when applying the patch
I am not getting this white space errors in my env, and also above QA report doesn't show
this, which way you are trying to apply the patch?

> Implement RM Container Reuse Requestor to handle the reuse containers for resource requests
> -------------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-6773
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6773
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: applicationmaster, mrv2
>            Reporter: Devaraj K
>            Assignee: Devaraj K
>         Attachments: MAPREDUCE-6773-v0.patch, MAPREDUCE-6773-v1.patch, MAPREDUCE-6773-v2.patch
> Add RM Container Reuse Requestor which handles the reuse containers against the Job reource

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org

View raw message