hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Vasudev (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3542) Re-factor support for CPU as a resource using the new ResourceHandler mechanism
Date Mon, 07 Dec 2015 15:26:11 GMT

     [ https://issues.apache.org/jira/browse/YARN-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Varun Vasudev updated YARN-3542:
    Attachment: YARN-3542.003.patch

bq.    Move the implementation from CgroupsLCEResourcesHandler into the new resource handler.
Mark the former as deprecated.


bq.    The existing config setting remains for backward compatibility - when CgroupsLCEResourcesHandler
is specified as the resource handler class, use the new resource handler implementation (
handled in ResourceHandlerModule ) . The existing integration in LinuxContainerExecutor can
probably be removed at this point.


bq.    Config behavior : If the existing config setting and the new config setting are both
explicitly specified and are incompatible, bail out ( this might be a bit tricky depending
on how defaults are handled for these settings )

This doesn't apply - there are no conflicting configs - the only issue was if both were specified,
but with the latest patch - it ends up creating only one object.

bq.    LOG.debug() statements should be guarded by a config check, IMO.

Guarded the debug statement that does some string manipulation. The others just print literal
strings - there should be little cost to them.

bq.    YarnConfiguration changes : Disk/Network configs define a prefix ( e.g NM_DISK_RESOURCE_PREFIX
) - maybe something similar should be used for CPU ?


> Re-factor support for CPU as a resource using the new ResourceHandler mechanism
> -------------------------------------------------------------------------------
>                 Key: YARN-3542
>                 URL: https://issues.apache.org/jira/browse/YARN-3542
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Sidharta Seethana
>            Assignee: Sidharta Seethana
>            Priority: Critical
>         Attachments: YARN-3542.001.patch, YARN-3542.002.patch, YARN-3542.003.patch
> In YARN-3443 , a new ResourceHandler mechanism was added which enabled easier addition
of new resource types in the nodemanager (this was used for network as a resource - See YARN-2140
). We should refactor the existing CPU implementation ( LinuxContainerExecutor/CgroupsLCEResourcesHandler
) using the new ResourceHandler mechanism. 

This message was sent by Atlassian JIRA

View raw message