hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bibin A Chundatt (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-14451) Deadlock in NativeIO
Date Wed, 24 May 2017 20:09:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16023448#comment-16023448
] 

Bibin A Chundatt edited comment on HADOOP-14451 at 5/24/17 8:08 PM:
--------------------------------------------------------------------

Thank you [~ajithshetty] .Analysis makes sense to me. The same could cause dead lock.
Tried a sample program to simulate.
{code}
		Thread one=new Thread(){
			@Override
			public void run() {
				NativeIO.isAvailable();
			}
		};		
		Thread two=new Thread(){
			@Override
			public void run() {
				NativeIO.POSIX.isAvailable();
		}
		};
		two.start();	
		one.start();
		one.join();
		two.join();
{code}
*stack trace*
{code}
"Thread-1" #9 prio=5 os_prio=0 tid=0x00007f1b90c5f000 nid=0x6843 in Object.wait() [0x00007f1b80ba3000]
   java.lang.Thread.State: RUNNABLE
	at org.apache.hadoop.io.nativeio.NativeIO.initNative(Native Method)
	at org.apache.hadoop.io.nativeio.NativeIO.<clinit>(NativeIO.java:644)
	at org.apache.hadoop.io.nativeio.TestNative$1.run(TestNative.java:8)

"Thread-2" #10 prio=5 os_prio=0 tid=0x00007f1b90c58000 nid=0x6842 in Object.wait() [0x00007f1b80ca4000]
   java.lang.Thread.State: RUNNABLE
	at org.apache.hadoop.io.nativeio.NativeIO$POSIX.<clinit>(NativeIO.java:184)
	at org.apache.hadoop.io.nativeio.TestNative$2.run(TestNative.java:15)
{code}

Affected versions are *2.8.0 and hadoop3*


cc/ [~cmccabe_impala_fa3f]  


was (Author: bibinchundatt):
Thank you [~ajithshetty] .Analysis makes sense to me. The same could cause dead lock.
Tried a sample program to simulate.
{code}
		Thread one=new Thread(){
			@Override
			public void run() {
				NativeIO.isAvailable();
			}
		};		
		Thread two=new Thread(){
			@Override
			public void run() {
				NativeIO.POSIX.isAvailable();
		}
		};
		two.start();	
		one.start();
		one.join();
		two.join();
{code}
*stack trace*
{code}
"Thread-1" #9 prio=5 os_prio=0 tid=0x00007f1b90c5f000 nid=0x6843 in Object.wait() [0x00007f1b80ba3000]
   java.lang.Thread.State: RUNNABLE
	at org.apache.hadoop.io.nativeio.NativeIO.initNative(Native Method)
	at org.apache.hadoop.io.nativeio.NativeIO.<clinit>(NativeIO.java:644)
	at org.apache.hadoop.io.nativeio.TestNative$1.run(TestNative.java:8)

"Thread-2" #10 prio=5 os_prio=0 tid=0x00007f1b90c58000 nid=0x6842 in Object.wait() [0x00007f1b80ca4000]
   java.lang.Thread.State: RUNNABLE
	at org.apache.hadoop.io.nativeio.NativeIO$POSIX.<clinit>(NativeIO.java:184)
	at org.apache.hadoop.io.nativeio.TestNative$2.run(TestNative.java:15)
{code}

Affected versions are *2.8.0 and hadoop3*

Another solution  could be to check  {{if(NativeIO.isAvailable())}} in {{FadvisedFileRegion#transferSuccessful}}
before  {code}NativeIO.POSIX.getCacheManipulator().posixFadviseIf
Possible(identifier,
            fd, getPosition(), getCount(), POSIX_FADV_DONTNEED);} {code}

cc/ [~cmccabe_impala_fa3f]  

> Deadlock in NativeIO
> --------------------
>
>                 Key: HADOOP-14451
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14451
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.8.0, 3.0.0-alpha1
>            Reporter: Ajith S
>            Assignee: Ajith S
>            Priority: Blocker
>         Attachments: Nodemanager.jstack
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message