hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shane Kumpf (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
Date Thu, 04 May 2017 17:43:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997127#comment-15997127
] 

Shane Kumpf commented on YARN-6515:
-----------------------------------

Thank you [~Naganarasimha] and [~ajisakaa] for bringing up the cgroup mount path hardcode
issue. 

I agree with [~miklos.szegedi@cloudera.com] that using YarnConfiguration.NM_LINUX_CONTAINER_CGROUPS_MOUNT_PATH
will require a documentation update due to the explicit wording of when it used. However,
the setting does seem to be appropriate for this use and I think introducing a new configuration
would be superfluous.

It would be good to remove the CGROUPS_ROOT_DIRECTORY variable all together and add a default
value  to YarnConfiguration ("/sys/fs/cgroup" to maintain backward compatibility). This will
require an update to TestDockerContainerRuntime as well.

{quote} Also, the location inside the container might still need to be hard coded or configured
by a separate entry to support isolation better.{quote}

IMO, I don't think we should change behavior here and allow the mount point in the container
to differ from the source (i.e. /sys/fs/cgroup should be bind mounted at /sys/fs/cgroup in
the container, as we do today). I believe we can remove this automatic cgroup mount and address
the need for different src/dest paths once YARN-5534 is in.

> Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
> ------------------------------------------------------------
>
>                 Key: YARN-6515
>                 URL: https://issues.apache.org/jira/browse/YARN-6515
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>         Attachments: YARN-6515.001.patch
>
>
> 5 find bugs issue was reported NM project as part of the YARN-4166 [build| https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html]
> Issue 1: 
> 	org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
is a mutable collection which should be package protected
> Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) 
> In class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics
> Field org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> At ContainerMetrics.java:\[line 134\]
> Issue 2:
> 	org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details) 
> In class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer
> In method org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> Field org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources
> At ContainerLocalizer.java:\[line 334\]
> Issue 3: 
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details) 
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> Field org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers
> At NodeStatusUpdaterImpl.java:\[line 721\]
> Issue 4: 
> Hard coded reference to an absolute pathname in org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details) 
> In class org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime
> In method org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> File name /sys/fs/cgroup
> At DockerLinuxContainerRuntime.java:\[line 455\]
> Useless object stored in variable removedNullContainers of method org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Bug type UC_USELESS_OBJECT (click for details) 
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Value removedNullContainers
> Type java.util.HashSet
> At NodeStatusUpdaterImpl.java:\[line 644\]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message