hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11851) getGlobalJNIEnv() may deadlock if exception is thrown
Date Thu, 18 May 2017 21:49:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016510#comment-16016510
] 

Hadoop QA commented on HDFS-11851:
----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 48s{color} | {color:blue}
Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  0s{color} | {color:red}
The patch doesn't appear to include any new or modified tests. Please justify why no new tests
are needed for this patch. Also please list what manual steps were performed to verify this
patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 37s{color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 16s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 12s{color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 14s{color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 11s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 13s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green}  0m 13s{color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 13s{color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 11s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 13s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m  0s{color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 39s{color} | {color:green}
hadoop-hdfs-native-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 21s{color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 59s{color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11851 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12868818/HDFS-11851.000.patch
|
| Optional Tests |  asflicense  compile  cc  mvnsite  javac  unit  |
| uname | Linux ec3c446a5b22 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 40e6a85 |
| Default Java | 1.8.0_131 |
|  Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/19499/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
|
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/19499/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> getGlobalJNIEnv() may deadlock if exception is thrown
> -----------------------------------------------------
>
>                 Key: HDFS-11851
>                 URL: https://issues.apache.org/jira/browse/HDFS-11851
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: libhdfs
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Henry Robinson
>            Assignee: Sailesh Mukil
>            Priority: Blocker
>         Attachments: HDFS-11851.000.patch
>
>
> HDFS-11529 introduced a deadlock into {{getGlobalJNIEnv()}} if an exception is thrown.
{{getGlobalJNIEnv()}} holds {{jvmMutex}}, but {{printExceptionAndFree()}} will eventually
try to acquire that lock in {{setTLSExceptionStrings()}}.
> The exception might get caught from {{loadFileSystems}}:
> {code}
> jthr = invokeMethod(env, NULL, STATIC, NULL,
>                          "org/apache/hadoop/fs/FileSystem",
>                          "loadFileSystems", "()V");
>         if (jthr) {
>             printExceptionAndFree(env, jthr, PRINT_EXC_ALL, "loadFileSystems");
>         }
>     }
> {code}
> and here's the relevant parts of the stack trace from where I call this API in Impala,
which uses {{libhdfs}}:
> {code}
> #0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
> #1  0x00007ffff4a8d657 in _L_lock_909 () from /lib/x86_64-linux-gnu/libpthread.so.0
> #2  0x00007ffff4a8d480 in __GI___pthread_mutex_lock (mutex=0x47ce960 <jvmMutex>)
at ../nptl/pthread_mutex_lock.c:79
> #3  0x0000000002f06056 in mutexLock (m=<optimized out>) at /data/2/jenkins/workspace/impala-hadoop-dependency/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/os/posix/mutexes.c:28
> #4  0x0000000002efe817 in setTLSExceptionStrings (rootCause=0x0, stackTrace=0x0) at /data/2/jenkins/workspace/impala-hadoop-dependency/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/jni_helper.c:581
> #5  0x0000000002f065d7 in printExceptionAndFreeV (env=0x513c1e8, exc=0x508a8c0, noPrintFlags=<optimized
out>, fmt=0x34349cf "loadFileSystems", ap=0x7fffffffb660)
>     at /data/2/jenkins/workspace/impala-hadoop-dependency/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/exception.c:183
> #6  0x0000000002f0683d in printExceptionAndFree (env=<optimized out>, exc=<optimized
out>, noPrintFlags=<optimized out>, fmt=<optimized out>)
>     at /data/2/jenkins/workspace/impala-hadoop-dependency/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/exception.c:213
> #7  0x0000000002eff60f in getGlobalJNIEnv () at /data/2/jenkins/workspace/impala-hadoop-dependency/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/native/libhdfs/jni_helper.c:463
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message