hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amit Nithian (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HDFS-1542) Deadlock in Configuration.writeXml when serialized form is larger than one DFS block
Date Mon, 27 Dec 2010 17:04:49 GMT

     [ https://issues.apache.org/jira/browse/HDFS-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amit Nithian updated HDFS-1542:
-------------------------------

    Attachment: hdfs1542_cdh3b3.txt

Todd,

Thanks for this patch. I took it and applied it to the CDH3b3 and have seen much better results
from my jobs that were previously deadlocking. Attached is the same patch except applied to
the CDH3b3 source for those who are using this and seeing the deadlock.

Thanks!
Amit

> Deadlock in Configuration.writeXml when serialized form is larger than one DFS block
> ------------------------------------------------------------------------------------
>
>                 Key: HDFS-1542
>                 URL: https://issues.apache.org/jira/browse/HDFS-1542
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>    Affects Versions: 0.22.0, 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Critical
>         Attachments: deadlock.txt, hdfs-1542.txt, hdfs1542_cdh3b3.txt, Test.java
>
>
> Configuration.writeXml holds a lock on itself and then writes the XML to an output stream,
during which DFSOutputStream will try to get a lock on ackQueue/dataQueue. Meanwihle the DataStreamer
thread will call functions like conf.getInt() and deadlock against the other thread, since
it could be the same conf object.
> This causes a deterministic deadlock whenever the serialized form is larger than block
size.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message