hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3139) DistributedFileSystem.close() deadlock and FileSystem.closeAll() warning
Date Thu, 03 Apr 2008 21:41:24 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tsz Wo (Nicholas), SZE updated HADOOP-3139:
-------------------------------------------

    Attachment: 3139_20080403.patch

3139_20080403.patch:
In HADOOP-3003, I added consistency check in FileSystem.Cache.closeAll() to check whether
the cache key is consistent with the stored fs (with the conf returned by fs.getConf()). 
However, a conf can be shared by several FileSystem objects and some other object like JobTracker.
 Therefore, the consistency check makes no sense since the conf will be modified from time
to time.  I remove the consistency check in this patch.  Indeed, the conf shouldn't be shared
between FileSystem.  We should fix it in another issue.

> DistributedFileSystem.close() deadlock and FileSystem.closeAll() warning
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-3139
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3139
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
>            Priority: Blocker
>             Fix For: 0.17.0
>
>         Attachments: 3139_20080331.patch, 3139_20080402b.patch, 3139_20080403.patch
>
>
> Koji found the following:
> *DistributedFileSystem.close() deadlock*
> My dfs -ls hang. 
> Ctrl-Z showed a deadlock state.
> {noformat}
> "Thread-0":
>         at org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:190)
>         - waiting to lock <0xedde8788> (a org.apache.hadoop.dfs.DistributedFileSystem)
>         at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1231)
>         - locked <0xee0baf88> (a org.apache.hadoop.fs.FileSystem$Cache)
>         at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:169)
>         at org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:154)
>         - locked <0xee0bae40> (a org.apache.hadoop.fs.FileSystem$ClientFinalizer)
> "main":
>         at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:1201)
>         - waiting to lock <0xee0baf88> (a org.apache.hadoop.fs.FileSystem$Cache)
>         at org.apache.hadoop.fs.FileSystem.close(FileSystem.java:1085)
>         at org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:192)
>         - locked <0xedde8788> (a org.apache.hadoop.dfs.DistributedFileSystem)
>         at org.apache.hadoop.fs.FsShell.close(FsShell.java:1698)
>         at org.apache.hadoop.fs.FsShell.main(FsShell.java:1712)
> Found 1 deadlock.
> {noformat}
> *FileSystem.closeAll() warning*
> {noformat}
> 08/03/31 16:48:42 INFO fs.FileSystem: FileSystem.closeAll() threw an exception:
> java.io.IOException: HftpFileSystem(=org.apache.hadoop.dfs.HftpFileSystem@111111) and
> Key(=null@hftp://namenode-nn:4444) do not match.
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message