hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3818) Not possible to access a FileSystem from within a ShutdownHook
Date Sat, 26 Jul 2008 14:47:31 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12617184#action_12617184
] 

Steve Loughran commented on HADOOP-3818:
----------------------------------------

in SmartFrog we had fun with Sun JVM signal handling, which provides extra compatibility problems
with the clean-room OSS implementations and are pretty hard to test. 

Maybe the classname of the class to run as a filesystem hook could be configurable, so someoneone
could subclass and extend its shutdown code, rather than run race condition pain in shutdown.
Which, for anyone who has tried, is essentially non-debuggable and therefore to be feared.


> Not possible to access a FileSystem from within a ShutdownHook
> --------------------------------------------------------------
>
>                 Key: HADOOP-3818
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3818
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.17.1
>            Reporter: Rowan Nairn
>            Priority: Minor
>
> FileSystem uses addShutdownHook to close all FileSystems at exit.  This makes it impossible
to access a FileSystem from within your own ShutdownHook threads, say for deleting incomplete
output.  Using a pre-existing FileSystem object is unsafe since it may be closed by the time
the thread executes.  Using FileSystem.get(...) results in an exception:
> Exception in thread "Thread-10" java.lang.IllegalStateException: Shutdown in progress
> 	at java.lang.Shutdown.add(Shutdown.java:81)
> 	at java.lang.Runtime.addShutdownHook(Runtime.java:190)
> 	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1293)
> 	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:203)
> 	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:108)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message