hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Badger (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6930) Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager
Date Wed, 16 Aug 2017 16:01:07 GMT

    [ https://issues.apache.org/jira/browse/YARN-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128993#comment-16128993

Eric Badger commented on YARN-6930:

The findbugs isn't related to this JIRA, butI think it's something that we should fix. We
can just make {{CGROUPS_ROOT_DIRECTORY}} another NM config. 

@@ -31,8 +32,13 @@
 import org.apache.hadoop.yarn.server.nodemanager.containermanager.runtime.ContainerRuntime;
 import org.apache.hadoop.yarn.server.nodemanager.containermanager.runtime.ContainerRuntimeContext;
+import java.util.Arrays;
+import java.util.List;
 import java.util.Map;
+import static org.apache.hadoop.yarn.conf.YarnConfiguration.DEFAULT_LINUX_CONTAINER_RUNTIME_ALLOWED_RUNTIMES;
+import static org.apache.hadoop.yarn.conf.YarnConfiguration.LINUX_CONTAINER_RUNTIME_ALLOWED_RUNTIMES;
Not sure of the convention here. Is it ok to import static variables one by one or should
we just import {{YarnConfiguration}} and reference the variables specifically?

@@ -50,25 +56,35 @@
   private DefaultLinuxContainerRuntime defaultLinuxContainerRuntime;
   private DockerLinuxContainerRuntime dockerLinuxContainerRuntime;
   private JavaSandboxLinuxContainerRuntime javaSandboxLinuxContainerRuntime;
+  private List<String> allowedRuntimes;
   public void initialize(Configuration conf)
       throws ContainerExecutionException {
+    allowedRuntimes = Arrays.asList(
     PrivilegedOperationExecutor privilegedOperationExecutor =
     defaultLinuxContainerRuntime = new DefaultLinuxContainerRuntime(
-    defaultLinuxContainerRuntime.initialize(conf);
+    if (isRuntimeAllowed(defaultLinuxContainerRuntime)) {
+      defaultLinuxContainerRuntime.initialize(conf);
+    }
     dockerLinuxContainerRuntime = new DockerLinuxContainerRuntime(
-    dockerLinuxContainerRuntime.initialize(conf);
+    if (isRuntimeAllowed(dockerLinuxContainerRuntime)) {
+      dockerLinuxContainerRuntime.initialize(conf);
+    }
     javaSandboxLinuxContainerRuntime = new JavaSandboxLinuxContainerRuntime(
-    javaSandboxLinuxContainerRuntime.initialize(conf);
+    if (isRuntimeAllowed(javaSandboxLinuxContainerRuntime)) {
+      javaSandboxLinuxContainerRuntime.initialize(conf);
+    }
It would save us a little bit of time and memory {{isRuntimeAllowed}} took the class name
directly. That way we could check to see if the runtime is allowed before we allocate it.
I also wonder whether it would be better/cleaner to iterate through {{allowedRuntimes}} and
allocate the new objects by class name via reflection instead of hard-coding all of them.
There's probably a reason why this is a terrible idea, but it's a thought. 

> Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager
> --------------------------------------------------------------------------------------------
>                 Key: YARN-6930
>                 URL: https://issues.apache.org/jira/browse/YARN-6930
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Shane Kumpf
>         Attachments: YARN-6930.001.patch, YARN-6930.002.patch, YARN-6930.003.patch
> Today, in the java land, all LinuxContainerRuntimes are always enabled when using LinuxContainerExecutor
and the user can simply invoke anything that he/she wants - default, docker, java-sandbox.
> We should have a way for admins to explicitly enable only specific runtimes that he/she
decides for the cluster. And by default, we should have everything other than the default
one disabled.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message