hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Ferguson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4351) Make ContainersMonitor pluggable
Date Fri, 13 Jul 2012 01:09:35 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413404#comment-13413404
] 

Andrew Ferguson commented on MAPREDUCE-4351:
--------------------------------------------

indeed, ContainersMonitor is not a great name for the current class. in my mind, it has four
jobs:

1) monitor resource usage
2) enforce resource limits
3) notify YARN of killed or failed tasks
4) delete temporary files created by container executor

while I see arguments to split each of these jobs into a separate class, I also see reasons
to keep them together (a la the current ContainersMonitor): jobs #1 and #3 are similar (poll
/proc), jobs #1 and #2 are related (in the current implementation, the results of #1 are used
for #2), and #2 and #4 are related, as you may need to revert some resource settings after
the job is complete.

thoughts?

thanks!
                
> Make ContainersMonitor pluggable
> --------------------------------
>
>                 Key: MAPREDUCE-4351
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4351
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: mrv2, nodemanager
>            Reporter: Andrew Ferguson
>            Assignee: Andrew Ferguson
>         Attachments: MAPREDUCE-4351-v1.patch, MAPREDUCE-4351-v2.patch, MAPREDUCE-4351-v3.patch,
MAPREDUCE-4351-v4.patch, MAPREDUCE-4351-v4.patch, MAPREDUCE-4351-v5.patch, MAPREDUCE-4351-v5.patch,
MAPREDUCE-4351-v6.patch
>
>
> Make the existing ContainersManager pluggable, just as the ContainerExecutor is currently.
This will allow us to add container resource enforcement using other techniques (such as cgroups)
in an extensible fashion.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message