hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sidharta Seethana (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3542) Re-factor support for CPU as a resource using the new ResourceHandler mechanism
Date Mon, 07 Dec 2015 08:31:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15044562#comment-15044562

Sidharta Seethana commented on YARN-3542:

Given that one of the goals of this JIRA/patch is to deprecate the current implementation,
how does this approach sound? 

 * Move the implementation from {{CgroupsLCEResourcesHandler}} into the new resource handler.
Mark the former as deprecated.
 * The existing config setting remains for backward compatibility - when {{CgroupsLCEResourcesHandler}}
is specified as the resource handler class, use the new resource handler implementation (
handled in {{ResourceHandlerModule}} ) . The existing integration in {{LinuxContainerExecutor}}
can probably be removed at this point. 
 * Config behavior : If the existing config setting and the new config setting are both explicitly
specified and are incompatible, bail out ( this might be a bit tricky depending on how defaults
are handled for these settings )

A couple of other minor comments based on a first look : 
 * LOG.debug() statements should be guarded by a config check, IMO. 
 * YarnConfiguration changes : Disk/Network configs define a prefix ( e.g NM_DISK_RESOURCE_PREFIX
) - maybe something similar should be used for CPU ? 

> Re-factor support for CPU as a resource using the new ResourceHandler mechanism
> -------------------------------------------------------------------------------
>                 Key: YARN-3542
>                 URL: https://issues.apache.org/jira/browse/YARN-3542
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Sidharta Seethana
>            Assignee: Sidharta Seethana
>            Priority: Critical
>         Attachments: YARN-3542.001.patch, YARN-3542.002.patch
> In YARN-3443 , a new ResourceHandler mechanism was added which enabled easier addition
of new resource types in the nodemanager (this was used for network as a resource - See YARN-2140
). We should refactor the existing CPU implementation ( LinuxContainerExecutor/CgroupsLCEResourcesHandler
) using the new ResourceHandler mechanism. 

This message was sent by Atlassian JIRA

View raw message