hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jian Fang <jian.fang.subscr...@gmail.com>
Subject Re: ClientFinalizer shutdown hook in FileSystem caused problems
Date Thu, 11 Apr 2013 23:33:05 GMT
Ok, I figured out the solution. I need to make sure my own shutdown hook
has higher priority than the one in FileSystem.



On Thu, Apr 11, 2013 at 12:08 PM, Jian Fang
<jian.fang.subscribe@gmail.com>wrote:

> Hi,
>
> We have some customized code to flush data during hadoop shutdown process,
> but we run into the following errors with Hadoop 2.0.3-alpha
>
> Exception in thread "Thread-3" java.lang.ExceptionInInitializerError
>         at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2323)
>         at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2295)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:319)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:165)
>         ...our custom code to check file system to flush
> Caused by: java.lang.IllegalStateException: Shutdown in progress
>         at
> java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:39)
>         at java.lang.Runtime.addShutdownHook(Runtime.java:192)
>         at
> org.apache.hadoop.util.ShutdownHookManager.<clinit>(ShutdownHookManager.java:47)
>
> Seems the Cache class in the FileSystem.java has the following logic
>
>
>          // now insert the new file system into the map
>         if (map.isEmpty() ) {
>           ShutdownHookManager.get().addShutdownHook(clientFinalizer,
> SHUTDOWN_HOOK_PRIORITY);
>         }
>
> What happened is that the above code was called in the shutdown process,
> thus, led to java.lang.IllegalStateException.
>
> I wonder why the shutdown hook was implemented in this way, why it was not
> added when the Cache class was initialized? How should I fix this problem
> then?
>
> Thanks,
>
> Jian
>

Mime
View raw message