hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Vasudev (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6623) Add support to turn off launching privileged containers in the container-executor
Date Tue, 17 Oct 2017 10:30:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16207264#comment-16207264
] 

Varun Vasudev commented on YARN-6623:
-------------------------------------

[~eyang] - the tests did run as part of the trunk pre-commit build. The test reports are lost
now but it will appear under the section (root) in the test report.

The tests seem to pass for me on the Mac and an Ubuntu VM. What's the environment you're using?
Maybe you should open a new ticket with details? 

Here's the result I got -
Mac
{noformat}
Varuns-MacBook-Pro:hadoop-yarn-server-nodemanager varun$ mvn test -Dtest=cetest -Pnative
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Hadoop YARN NodeManager 3.1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-yarn-server-nodemanager
---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO]
[INFO] --- hadoop-maven-plugins:3.1.0-SNAPSHOT:protoc (compile-protoc) @ hadoop-yarn-server-nodemanager
---
[INFO] No changes detected in protoc files, skipping generation.
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hadoop-yarn-server-nodemanager
---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hadoop-yarn-server-nodemanager
---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hadoop-yarn-server-nodemanager
---
[INFO] Compiling 3 source files to /Users/varun/Workspace/apache/committer-rw/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
[INFO]
[INFO] --- hadoop-maven-plugins:3.1.0-SNAPSHOT:cmake-compile (cmake-compile) @ hadoop-yarn-server-nodemanager
---
[INFO] Running cmake /Users/varun/Workspace/apache/committer-rw/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src
-DHADOOP_CONF_DIR=../etc/hadoop -DJVM_ARCH_DATA_MODEL=64 -G Unix Makefiles
[INFO] with extra environment variables {
  CFLAGS = ''
}
[INFO] Running make -j 8 VERBOSE=1
[INFO] cmake compilation finished successfully in 205 millisecond(s).
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (make) @ hadoop-yarn-server-nodemanager ---
[INFO] Executing tasks

main:
     [copy] Copying 4 files to /Users/varun/Workspace/apache/committer-rw/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/test
[INFO] Executed tasks
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hadoop-yarn-server-nodemanager
---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 7 resources
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hadoop-yarn-server-nodemanager
---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ hadoop-yarn-server-nodemanager
---
[INFO]
[INFO] --- hadoop-maven-plugins:3.1.0-SNAPSHOT:cmake-test (test-container-executor) @ hadoop-yarn-server-nodemanager
---
[INFO]
[INFO] --- hadoop-maven-plugins:3.1.0-SNAPSHOT:cmake-test (cetest) @ hadoop-yarn-server-nodemanager
---
[INFO] -------------------------------------------------------
[INFO]  C M A K E B U I L D E R    T E S T
[INFO] -------------------------------------------------------
[INFO] cetest: running /Users/varun/Workspace/apache/committer-rw/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/test/cetest
--gtest_filter=-Perf. --gtest_output=xml:/Users/varun/Workspace/apache/committer-rw/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/surefire-reports/TEST-cetest.xml
[INFO] with extra environment variables {}
[INFO] STATUS: SUCCESS after 87 millisecond(s).
[INFO] -------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.263 s
[INFO] Finished at: 2017-10-17T15:43:32+05:30
[INFO] Final Memory: 35M/318M
[INFO] ------------------------------------------------------------------------
{noformat}

Ubuntu 16
{noformat}
varun@nm-u16:~/Workspace/apache/dev/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager$
mvn test -Dtest=cetest -Pnative
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Hadoop YARN NodeManager 3.1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-yarn-server-nodemanager
---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO]
[INFO] --- hadoop-maven-plugins:3.1.0-SNAPSHOT:protoc (compile-protoc) @ hadoop-yarn-server-nodemanager
---
[INFO] No changes detected in protoc files, skipping generation.
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hadoop-yarn-server-nodemanager
---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hadoop-yarn-server-nodemanager
---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hadoop-yarn-server-nodemanager
---
[INFO] Compiling 3 source files to /home/varun/Workspace/apache/dev/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes
[INFO]
[INFO] --- hadoop-maven-plugins:3.1.0-SNAPSHOT:cmake-compile (cmake-compile) @ hadoop-yarn-server-nodemanager
---
[INFO] Running cmake /home/varun/Workspace/apache/dev/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src
-DHADOOP_CONF_DIR=../etc/hadoop -DJVM_ARCH_DATA_MODEL=64 -G Unix Makefiles
[INFO] with extra environment variables {
  CFLAGS = ''
}
[INFO] Running make -j 1 VERBOSE=1
[INFO] cmake compilation finished successfully in 365 millisecond(s).
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (make) @ hadoop-yarn-server-nodemanager ---
[INFO] Executing tasks

main:
     [copy] Copying 4 files to /home/varun/Workspace/apache/dev/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/test
[INFO] Executed tasks
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hadoop-yarn-server-nodemanager
---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 7 resources
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hadoop-yarn-server-nodemanager
---
[INFO] Compiling 29 source files to /home/varun/Workspace/apache/dev/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/test-classes
[WARNING] /home/varun/Workspace/apache/dev/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java:
/home/varun/Workspace/apache/dev/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java
uses or overrides a deprecated API.
[WARNING] /home/varun/Workspace/apache/dev/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/TestLogAggregationService.java:
Recompile with -Xlint:deprecation for details.
[WARNING] /home/varun/Workspace/apache/dev/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestJavaSandboxLinuxContainerRuntime.java:
Some input files use unchecked or unsafe operations.
[WARNING] /home/varun/Workspace/apache/dev/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestJavaSandboxLinuxContainerRuntime.java:
Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ hadoop-yarn-server-nodemanager
---
[INFO]
[INFO] --- hadoop-maven-plugins:3.1.0-SNAPSHOT:cmake-test (test-container-executor) @ hadoop-yarn-server-nodemanager
---
[INFO]
[INFO] --- hadoop-maven-plugins:3.1.0-SNAPSHOT:cmake-test (cetest) @ hadoop-yarn-server-nodemanager
---
[INFO] -------------------------------------------------------
[INFO]  C M A K E B U I L D E R    T E S T
[INFO] -------------------------------------------------------
[INFO] cetest: running /home/varun/Workspace/apache/dev/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native/test/cetest
--gtest_filter=-Perf. --gtest_output=xml:/home/varun/Workspace/apache/dev/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/surefire-reports/TEST-cetest.xml
[INFO] with extra environment variables {}
[INFO] STATUS: SUCCESS after 64 millisecond(s).
[INFO] -------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.906 s
[INFO] Finished at: 2017-10-17T15:50:48+05:30
[INFO] Final Memory: 38M/121M
[INFO] ------------------------------------------------------------------------
{noformat}

> Add support to turn off launching privileged containers in the container-executor
> ---------------------------------------------------------------------------------
>
>                 Key: YARN-6623
>                 URL: https://issues.apache.org/jira/browse/YARN-6623
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>            Priority: Blocker
>             Fix For: 3.0.0
>
>         Attachments: YARN-6623-branch-2.013.patch, YARN-6623-branch-2.014.patch, YARN-6623-branch-2.015.patch,
YARN-6623.001.patch, YARN-6623.002.patch, YARN-6623.003.patch, YARN-6623.004.patch, YARN-6623.005.patch,
YARN-6623.006.patch, YARN-6623.007.patch, YARN-6623.008.patch, YARN-6623.009.patch, YARN-6623.010.patch,
YARN-6623.011.patch, YARN-6623.012.patch, YARN-6623.013.patch, cetest.stderr, cetest.stdout
>
>
> Currently, launching privileged containers is controlled by the NM. We should add a flag
to the container-executor.cfg allowing admins to disable launching privileged containers at
the container-executor level.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message