hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John George (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-2081) trying to read configuration in hdfs write path might result in deadlock
Date Fri, 17 Jun 2011 22:54:47 GMT
trying to read configuration in hdfs write path might result in deadlock
------------------------------------------------------------------------

                 Key: HDFS-2081
                 URL: https://issues.apache.org/jira/browse/HDFS-2081
             Project: Hadoop HDFS
          Issue Type: Bug
         Environment: MR-279 branch. This is specific to yarn since yarn tries to write the
configuration file to hdfs while holding the lock of the configuration file.
            Reporter: John George
            Assignee: John George


While trying to run yarn with 1024 block size, it looks as though the dfsclients are hung
waiting to write to DN. The reason is because yarn golds the configuration file lock and tries
to write to hdfs which in turn tries to read from the configuration which requires the same
lock.

The temporary fix is to make sure that dfsclient threads dont read the config in its write
path. There might be better long term solutions like read/write locks and this needs to be
investigated in another JIRA.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message