hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3818) Not possible to access a FileSystem from within a ShutdownHook
Date Fri, 25 Jul 2008 19:23:31 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12617018#action_12617018
] 

Tsz Wo (Nicholas), SZE commented on HADOOP-3818:
------------------------------------------------

A trick not sure it will work for you: First create files and call FileSystem.deleteOnExit(...)
for them.  Then, do rename at the end of the application.  If user terminates the process,
they file won't be renamed and will be deleted.

It will be more efficient if you put everything in a directory and do deleteOnExit and rename
on that directory.


> Not possible to access a FileSystem from within a ShutdownHook
> --------------------------------------------------------------
>
>                 Key: HADOOP-3818
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3818
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.17.1
>            Reporter: Rowan Nairn
>            Priority: Minor
>
> FileSystem uses addShutdownHook to close all FileSystems at exit.  This makes it impossible
to access a FileSystem from within your own ShutdownHook threads, say for deleting incomplete
output.  Using a pre-existing FileSystem object is unsafe since it may be closed by the time
the thread executes.  Using FileSystem.get(...) results in an exception:
> Exception in thread "Thread-10" java.lang.IllegalStateException: Shutdown in progress
> 	at java.lang.Shutdown.add(Shutdown.java:81)
> 	at java.lang.Runtime.addShutdownHook(Runtime.java:190)
> 	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1293)
> 	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:203)
> 	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:108)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message