Return-Path: Delivered-To: apmail-hadoop-core-dev-archive@www.apache.org Received: (qmail 74247 invoked from network); 1 Apr 2008 19:18:57 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 1 Apr 2008 19:18:57 -0000 Received: (qmail 80056 invoked by uid 500); 1 Apr 2008 19:18:52 -0000 Delivered-To: apmail-hadoop-core-dev-archive@hadoop.apache.org Received: (qmail 80027 invoked by uid 500); 1 Apr 2008 19:18:52 -0000 Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-dev@hadoop.apache.org Received: (qmail 80018 invoked by uid 99); 1 Apr 2008 19:18:52 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Apr 2008 12:18:52 -0700 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Apr 2008 19:18:09 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 31E70234C0B1 for ; Tue, 1 Apr 2008 12:16:24 -0700 (PDT) Message-ID: <797018315.1207077384199.JavaMail.jira@brutus> Date: Tue, 1 Apr 2008 12:16:24 -0700 (PDT) From: "Tsz Wo (Nicholas), SZE (JIRA)" To: core-dev@hadoop.apache.org Subject: [jira] Updated: (HADOOP-3139) DistributedFileSystem.close() deadlock and FileSystem.closeAll() warning In-Reply-To: <185404300.1206994225371.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HADOOP-3139: ------------------------------------------- Description: Koji found the following: *DistributedFileSystem.close() deadlock* My dfs -ls hang. Ctrl-Z showed a deadlock state. {noformat} "Thread-0": at org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:190) - waiting to lock <0xedde8788> (a org.apache.hadoop.dfs.DistributedFileSystem) at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1231) - locked <0xee0baf88> (a org.apache.hadoop.fs.FileSystem$Cache) at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:169) at org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:154) - locked <0xee0bae40> (a org.apache.hadoop.fs.FileSystem$ClientFinalizer) "main": at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:1201) - waiting to lock <0xee0baf88> (a org.apache.hadoop.fs.FileSystem$Cache) at org.apache.hadoop.fs.FileSystem.close(FileSystem.java:1085) at org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:192) - locked <0xedde8788> (a org.apache.hadoop.dfs.DistributedFileSystem) at org.apache.hadoop.fs.FsShell.close(FsShell.java:1698) at org.apache.hadoop.fs.FsShell.main(FsShell.java:1712) Found 1 deadlock. {noformat} *FileSystem.closeAll() warning* {noformat} 08/03/31 16:48:42 INFO fs.FileSystem: FileSystem.closeAll() threw an exception: java.io.IOException: HftpFileSystem(=org.apache.hadoop.dfs.HftpFileSystem@111111) and Key(=null@hftp://namenode-nn:4444) do not match. {noformat} was: Koji found the following: My dfs -ls hang. Ctrl-Z showed a deadlock state. {noformat} "Thread-0": at org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:190) - waiting to lock <0xedde8788> (a org.apache.hadoop.dfs.DistributedFileSystem) at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1231) - locked <0xee0baf88> (a org.apache.hadoop.fs.FileSystem$Cache) at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:169) at org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:154) - locked <0xee0bae40> (a org.apache.hadoop.fs.FileSystem$ClientFinalizer) "main": at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:1201) - waiting to lock <0xee0baf88> (a org.apache.hadoop.fs.FileSystem$Cache) at org.apache.hadoop.fs.FileSystem.close(FileSystem.java:1085) at org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:192) - locked <0xedde8788> (a org.apache.hadoop.dfs.DistributedFileSystem) at org.apache.hadoop.fs.FsShell.close(FsShell.java:1698) at org.apache.hadoop.fs.FsShell.main(FsShell.java:1712) Found 1 deadlock. {noformat} Summary: DistributedFileSystem.close() deadlock and FileSystem.closeAll() warning (was: DistributedFileSystem.close() introduces a deadlock) > DistributedFileSystem.close() deadlock and FileSystem.closeAll() warning > ------------------------------------------------------------------------ > > Key: HADOOP-3139 > URL: https://issues.apache.org/jira/browse/HADOOP-3139 > Project: Hadoop Core > Issue Type: Bug > Components: dfs > Reporter: Tsz Wo (Nicholas), SZE > Assignee: Tsz Wo (Nicholas), SZE > Priority: Blocker > Fix For: 0.17.0 > > Attachments: 3139_20080331.patch > > > Koji found the following: > *DistributedFileSystem.close() deadlock* > My dfs -ls hang. > Ctrl-Z showed a deadlock state. > {noformat} > "Thread-0": > at org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:190) > - waiting to lock <0xedde8788> (a org.apache.hadoop.dfs.DistributedFileSystem) > at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1231) > - locked <0xee0baf88> (a org.apache.hadoop.fs.FileSystem$Cache) > at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:169) > at org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:154) > - locked <0xee0bae40> (a org.apache.hadoop.fs.FileSystem$ClientFinalizer) > "main": > at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:1201) > - waiting to lock <0xee0baf88> (a org.apache.hadoop.fs.FileSystem$Cache) > at org.apache.hadoop.fs.FileSystem.close(FileSystem.java:1085) > at org.apache.hadoop.dfs.DistributedFileSystem.close(DistributedFileSystem.java:192) > - locked <0xedde8788> (a org.apache.hadoop.dfs.DistributedFileSystem) > at org.apache.hadoop.fs.FsShell.close(FsShell.java:1698) > at org.apache.hadoop.fs.FsShell.main(FsShell.java:1712) > Found 1 deadlock. > {noformat} > *FileSystem.closeAll() warning* > {noformat} > 08/03/31 16:48:42 INFO fs.FileSystem: FileSystem.closeAll() threw an exception: > java.io.IOException: HftpFileSystem(=org.apache.hadoop.dfs.HftpFileSystem@111111) and > Key(=null@hftp://namenode-nn:4444) do not match. > {noformat} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.