Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <1586879351.1207549825058.JavaMail.jira@brutus>
Date: Sun, 6 Apr 2008 23:30:25 -0700 (PDT)
From: "dhruba borthakur (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-3197) Deadlock in DFCClient
In-Reply-To: <1474442010.1207395205028.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HADOOP-3197?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D125=
86250#action_12586250 ]=20

dhruba borthakur commented on HADOOP-3197:
------------------------------------------

It is possible that you have a slow datanode that is participating in this =
pipeline only. This might have caused the DataStreamer thread to block on t=
he socketWrite. You would also see that this thread is marked as RUNNABLE, =
probably because it is writing out its bufefr slowly.

> Deadlock in DFCClient
> ---------------------
>
>                 Key: HADOOP-3197
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3197
>             Project: Hadoop Core
>          Issue Type: Bug
>    Affects Versions: 0.16.1
>            Reporter: Andr=C3=A9 Martin
>
> The DFS Client hangs - attached the thread dump - looks like a dead lock =
to me...
> {noformat}
> "ResponseProcessor for block blk_-7822837545361798562" prio=3D10 tid=3D0x=
00002aab993dcc00 nid=3D0x5241 waiting for monitor entry [0x000000004365e000=
..0x000000004365ecc0]
>    java.lang.Thread.State: BLOCKED (on object monitor)
> =09at org.apache.hadoop.dfs.DFSClient$DFSOutputStream$ResponseProcessor.r=
un(DFSClient.java:1771)
> =09- waiting to lock <0x00002aaaecf2dd08> (a java.util.LinkedList)
> "DataStreamer for file /seDNS/mapred-out/18A59C65A91D44E5BA24785DF103D178=
1BB0137E.cache.new block blk_-7822837545361798562" prio=3D10 tid=3D0x00002a=
ab96a46000 nid=3D0x523f runnable [0x000000004345c000..0x000000004345cc40]
>    java.lang.Thread.State: RUNNABLE
> =09at java.net.SocketOutputStream.socketWrite0(Native Method)
> =09at java.net.SocketOutputStream.socketWrite(Unknown Source)
> =09at java.net.SocketOutputStream.write(Unknown Source)
> =09at java.io.BufferedOutputStream.write(Unknown Source)
> =09- locked <0x00002aaaecf2ec50> (a java.io.BufferedOutputStream)
> =09at java.io.DataOutputStream.write(Unknown Source)
> =09- locked <0x00002aaaecf2ec20> (a java.io.DataOutputStream)
> =09at org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DF=
SClient.java:1623)
> =09- locked <0x00002aaaecf2dd08> (a java.util.LinkedList)
> "BackupJobQueuesThread" prio=3D10 tid=3D0x00002aab94b94000 nid=3D0x7cb2 w=
aiting for monitor entry [0x000000004244c000..0x000000004244cd40]
>    java.lang.Thread.State: BLOCKED (on object monitor)
> =09at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.writeChunk(DFSClien=
t.java:2117)
> =09- waiting to lock <0x00002aaaecf2dd08> (a java.util.LinkedList)
> =09at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSumm=
er.java:141)
> =09at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java=
:124)
> =09- locked <0x00002aaaecf2e670> (a org.apache.hadoop.dfs.DFSClient$DFSOu=
tputStream)
> =09at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:58)
> =09- locked <0x00002aaaecf2e670> (a org.apache.hadoop.dfs.DFSClient$DFSOu=
tputStream)
> =09at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataO=
utputStream.java:36)
> =09at java.io.DataOutputStream.writeBytes(Unknown Source)
> =09at sedns.serializer.file.FileSerializerServer.serializeJobQueuesAndCac=
he(FileSerializerServer.java:723)
> =09- locked <0x00002aaab430fec8> (a java.util.Collections$SynchronizedSet=
)
> =09at sedns.pastry.application.ServerApp$BackupJobListThread.run(ServerAp=
p.java:476)
> "org.apache.hadoop.dfs.DFSClient$LeaseChecker@3acafb56" daemon prio=3D10 =
tid=3D0x00002aab94bc7c00 nid=3D0x7ca7 waiting on condition [0x0000000041941=
000..0x0000000041941bc0]
>    java.lang.Thread.State: TIMED_WAITING (sleeping)
> =09at java.lang.Thread.sleep(Native Method)
> =09at org.apache.hadoop.dfs.DFSClient$LeaseChecker.run(DFSClient.java:597=
)
> =09at java.lang.Thread.run(Unknown Source)
> {noformat}
> =20

--=20
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.