Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 16 Aug 2017 16:01:07 +0000 (UTC)
From: "Eric Badger (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13091983.1501702524000.44812.1502899267480@Atlassian.JIRA>
In-Reply-To: <JIRA.13091983.1501702524000@Atlassian.JIRA>
References: <JIRA.13091983.1501702524000@Atlassian.JIRA> <JIRA.13091983.1501702524048@jira-lw-us.apache.org>
Subject: [jira] [Commented] (YARN-6930) Admins should be able to explicitly
 enable specific LinuxContainerRuntime in the NodeManager
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 16 Aug 2017 16:01:18 -0000


    [ https://issues.apache.org/jira/browse/YARN-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128993#comment-16128993 ] 

Eric Badger commented on YARN-6930:
-----------------------------------

The findbugs isn't related to this JIRA, butI think it's something that we should fix. We can just make {{CGROUPS_ROOT_DIRECTORY}} another NM config. 

DelegatingLinuxContainerRuntime.java
{noformat}
@@ -31,8 +32,13 @@
 import org.apache.hadoop.yarn.server.nodemanager.containermanager.runtime.ContainerRuntime;
 import org.apache.hadoop.yarn.server.nodemanager.containermanager.runtime.ContainerRuntimeContext;
 
+import java.util.Arrays;
+import java.util.List;
 import java.util.Map;
 
+import static org.apache.hadoop.yarn.conf.YarnConfiguration.DEFAULT_LINUX_CONTAINER_RUNTIME_ALLOWED_RUNTIMES;
+import static org.apache.hadoop.yarn.conf.YarnConfiguration.LINUX_CONTAINER_RUNTIME_ALLOWED_RUNTIMES;
+
{noformat}
Not sure of the convention here. Is it ok to import static variables one by one or should we just import {{YarnConfiguration}} and reference the variables specifically?

{noformat}
@@ -50,25 +56,35 @@
   private DefaultLinuxContainerRuntime defaultLinuxContainerRuntime;
   private DockerLinuxContainerRuntime dockerLinuxContainerRuntime;
   private JavaSandboxLinuxContainerRuntime javaSandboxLinuxContainerRuntime;
+  private List<String> allowedRuntimes;
 
   @Override
   public void initialize(Configuration conf)
       throws ContainerExecutionException {
+    allowedRuntimes = Arrays.asList(
+        conf.getTrimmedStrings(LINUX_CONTAINER_RUNTIME_ALLOWED_RUNTIMES,
+        DEFAULT_LINUX_CONTAINER_RUNTIME_ALLOWED_RUNTIMES));
     PrivilegedOperationExecutor privilegedOperationExecutor =
         PrivilegedOperationExecutor.getInstance(conf);
     defaultLinuxContainerRuntime = new DefaultLinuxContainerRuntime(
         privilegedOperationExecutor);
-    defaultLinuxContainerRuntime.initialize(conf);
+    if (isRuntimeAllowed(defaultLinuxContainerRuntime)) {
+      defaultLinuxContainerRuntime.initialize(conf);
+    }
     dockerLinuxContainerRuntime = new DockerLinuxContainerRuntime(
         privilegedOperationExecutor);
-    dockerLinuxContainerRuntime.initialize(conf);
+    if (isRuntimeAllowed(dockerLinuxContainerRuntime)) {
+      dockerLinuxContainerRuntime.initialize(conf);
+    }
     javaSandboxLinuxContainerRuntime = new JavaSandboxLinuxContainerRuntime(
         privilegedOperationExecutor);
-    javaSandboxLinuxContainerRuntime.initialize(conf);
+    if (isRuntimeAllowed(javaSandboxLinuxContainerRuntime)) {
+      javaSandboxLinuxContainerRuntime.initialize(conf);
+    }
{noformat}
It would save us a little bit of time and memory {{isRuntimeAllowed}} took the class name directly. That way we could check to see if the runtime is allowed before we allocate it. I also wonder whether it would be better/cleaner to iterate through {{allowedRuntimes}} and allocate the new objects by class name via reflection instead of hard-coding all of them. There's probably a reason why this is a terrible idea, but it's a thought. 


> Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager
> --------------------------------------------------------------------------------------------
>
>                 Key: YARN-6930
>                 URL: https://issues.apache.org/jira/browse/YARN-6930
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Shane Kumpf
>         Attachments: YARN-6930.001.patch, YARN-6930.002.patch, YARN-6930.003.patch
>
>
> Today, in the java land, all LinuxContainerRuntimes are always enabled when using LinuxContainerExecutor and the user can simply invoke anything that he/she wants - default, docker, java-sandbox.
> We should have a way for admins to explicitly enable only specific runtimes that he/she decides for the cluster. And by default, we should have everything other than the default one disabled.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org