hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (Jira)" <j...@apache.org>
Subject [jira] [Commented] (YARN-10495) make the rpath of container-executor configurable
Date Thu, 19 Nov 2020 18:53:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-10495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235658#comment-17235658
] 

Hadoop QA commented on YARN-10495:
----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime ||  Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m  9s{color} |  | {color:blue}
Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green}  0m  0s{color} |
 | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} codespell {color} | {color:blue}  0m  2s{color} |  |
{color:blue} codespell was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
 | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  0s{color} |  |
{color:red} The patch doesn't appear to include any new or modified tests. Please justify
why no new tests are needed for this patch. Also please list what manual steps were performed
to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  8m 20s{color} | [/branch-mvninstall-root.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/branch-mvninstall-root.txt]
| {color:red} root in trunk failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 10s{color} | [/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.txt]
| {color:red} hadoop-yarn-server-nodemanager in trunk failed with JDK Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 20s{color} | [/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.txt]
| {color:red} hadoop-yarn-server-nodemanager in trunk failed with JDK Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 48s{color} |
 | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 10m 32s{color} | [/branch-shadedclient.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/branch-shadedclient.txt]
| {color:red} branch has errors when building and testing our client artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 20s{color} | [/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.txt]
| {color:red} hadoop-yarn-server-nodemanager in trunk failed with JDK Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.
{color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 25s{color} | [/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.txt]
| {color:red} hadoop-yarn-server-nodemanager in trunk failed with JDK Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.
{color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 48s{color}
|  | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 23s{color} | [/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.txt]
| {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.
{color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red}  0m 23s{color} | [/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.txt]
| {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 23s{color} | [/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.txt]
| {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 23s{color} | [/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.txt]
| {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.
{color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red}  0m 23s{color} | [/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.txt]
| {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 23s{color} | [/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.txt]
| {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.
{color} |
| {color:green}+1{color} | {color:green} blanks {color} | {color:green}  0m  0s{color} | 
| {color:green} The patch has no blanks issues. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red}  0m 25s{color} | [/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt]
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  2s{color} |  | {color:green}
The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}  0m 25s{color}
|  | {color:green} patch has no errors when building and testing our client artifacts. {color}
|
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 23s{color} | [/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkUbuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.txt]
| {color:red} hadoop-yarn-server-nodemanager in the patch failed with JDK Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1.
{color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 31s{color} | [/results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10.txt]
| {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdkPrivateBuild-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10
with JDK Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 generated 196 new + 0 unchanged
- 0 fixed = 196 total (was 0) {color} |
|| || || || {color:brown} Other Tests {color} || ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 24m 53s{color} |  |
{color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  1m  4s{color} | [/results-asflicense.txt|https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/results-asflicense.txt]
| {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 57s{color} |  | {color:black}{color}
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10495 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/13015610/YARN-10495.001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient
codespell xml cc |
| uname | Linux d61467884eb1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 34aa6137bd890a565ace26be278a50f81b3dda20 |
| Default Java | Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9+11-Ubuntu-0ubuntu1.18.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_272-8u272-b10-0ubuntu1~18.04-b10 |
|  Test Results | https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/testReport/ |
| Max. process+thread count | 168 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager |
| Console output | https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/318/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |


This message was automatically generated.



> make the rpath of container-executor configurable
> -------------------------------------------------
>
>                 Key: YARN-10495
>                 URL: https://issues.apache.org/jira/browse/YARN-10495
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn
>            Reporter: angerszhu
>            Assignee: angerszhu
>            Priority: Major
>         Attachments: YARN-10495.001.patch
>
>
> In  https://issues.apache.org/jira/browse/YARN-9561 we add dependency on crypto to
container-executor, we meet a case that in our jenkins machine, we have libcrypto.so.1.0.0 
in shared lib env. but in our nodemanager machine we don't have  libcrypto.so.1.0.0  but *libcrypto.so.1.1.*
> We use a  internal custom dynamic link library environment /usr/lib/x86_64-linux-gnu
> and we build hadoop with parameter as blow
> {code:java}
>  -Drequire.openssl -Dbundle.openssl -Dopenssl.lib=/usr/lib/x86_64-linux-gnu
> {code}
>  
> Under jenkins machine shared lib library path /usr/lib/x86_64-linux-gun(where is libcrypto)
> {code:java}
> -rw-r--r-- 1 root root   240136 Nov 28  2014 libcroco-0.6.so.3.0.1
> -rw-r--r-- 1 root root    54550 Jun 18  2017 libcrypt.a
> -rw-r--r-- 1 root root  4306444 Sep 26  2019 libcrypto.a
> lrwxrwxrwx 1 root root       18 Sep 26  2019 libcrypto.so -> libcrypto.so.1.0.0
> -rw-r--r-- 1 root root  2070976 Sep 26  2019 libcrypto.so.1.0.0
> lrwxrwxrwx 1 root root       35 Jun 18  2017 libcrypt.so -> /lib/x86_64-linux-gnu/libcrypt.so.1
> -rw-r--r-- 1 root root      298 Jun 18  2017 libc.so
> {code}
>  
> Under nodemanager shared lib library path /usr/lib/x86_64-linux-gun(where is libcrypto)
> {code:java}
> -rw-r--r--  1 root root    55852 2��   7  2019 libcrypt.a
> -rw-r--r--  1 root root  4864244 9��  28  2019 libcrypto.a
> lrwxrwxrwx  1 root root       16 9��  28  2019 libcrypto.so -> libcrypto.so.1.1
> -rw-r--r--  1 root root  2504576 12�� 24  2019 libcrypto.so.1.0.2
> -rw-r--r--  1 root root  2715840 9��  28  2019 libcrypto.so.1.1
> lrwxrwxrwx  1 root root       35 2��   7  2019 libcrypt.so -> /lib/x86_64-linux-gnu/libcrypt.so.1
> -rw-r--r--  1 root root      298 2��   7  2019 libc.so
> {code}
>  We build container-executor with 
> The  libcrypto.so 's version is not same case error when we start nodemanager
>  
> {code:java}
> .. 3 more Caused by: org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.PrivilegedOperationException:
ExitCodeException exitCode=127: /home/hadoop/hadoop/bin/container-executor: error while loading
shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory
at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.PrivilegedOperationExecutor.executePrivilegedOperation(PrivilegedOperationExecutor.java:182)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.PrivilegedOperationExecutor.executePrivilegedOperation(PrivilegedOperationExecutor.java:208)
at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:306)
... 4 more Caused by: ExitCodeException exitCode=127: /home/hadoop/hadoop/bin/container-executor:
error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file:
No such file or directory at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008) at org.apache.hadoop.util.Shell.run(Shell.java:901)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213) at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.PrivilegedOperationExecutor.executePrivilegedOperation(PrivilegedOperationExecutor.java:154)
... 6 more 
> {code}
>  
> We should make RPATH of container-executor configurable to solve this problem 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message