hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chuan Liu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2190) Provide a Windows container executor that can limit memory and CPU
Date Sat, 16 Aug 2014 18:08:19 GMT

     [ https://issues.apache.org/jira/browse/YARN-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chuan Liu updated YARN-2190:

    Attachment: YARN-2190.3.patch

bq. 1. Where can I see that there are CPU/memory limits set on the job? ProcessExplorer?
I can see the memory limits in the latest process explorer; but not the cpu rate limit yet.

bq. 2. Please make sure the code continues to compile/run on Win7/SDK. I am still on Server

Yes. It can be build with Windows Server 2008 R2 with or without CPU limit support. 

bq. 3. task.c: Can you init wchar_t *end to NULL? In the if check after wcstol, might make
sense to add end == NULL || *end !=...
bq. 4. task.c: ParseCommandLine: Given that you're passing pointers to variables on stack,
you could as well assert that memory and vcore are != NULL.

Fixed in new patch.

bq. 5. Can you please specify the unit used for the memory/CPU limit?

Added more information in the help message.

bq. 6. Should we multiply first and then divide, to minimize precision loss?


bq. 7. Would you mind including a unittest for WindowsContainerExecutor? At this point it
will be a trivial test, but will likely grow over time. 

Added a new unit test.

bq. 8. Just to confirm, by default, we will still use the DefaultContainerExecutor on Windows,
right? And users can configure the WindowsContainerExecutor if they want? This sounds good
until we develop better understand of how new limits behave in production.

Yes. Default is not changed; and WindowsContainerExecutor is only an optional plugin.

> Provide a Windows container executor that can limit memory and CPU
> ------------------------------------------------------------------
>                 Key: YARN-2190
>                 URL: https://issues.apache.org/jira/browse/YARN-2190
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager
>            Reporter: Chuan Liu
>            Assignee: Chuan Liu
>         Attachments: YARN-2190-prototype.patch, YARN-2190.1.patch, YARN-2190.2.patch,
> Yarn default container executor on Windows does not set the resource limit on the containers
currently. The memory limit is enforced by a separate monitoring thread. The container implementation
on Windows uses Job Object right now. The latest Windows (8 or later) API allows CPU and memory
limits on the job objects. We want to create a Windows container executor that sets the limits
on job objects thus provides resource enforcement at OS level.
> http://msdn.microsoft.com/en-us/library/windows/desktop/ms686216(v=vs.85).aspx

This message was sent by Atlassian JIRA

View raw message