hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lukas Kairies <lukas.xtree...@googlemail.com>
Subject Re: Shutdown hook for FileSystems
Date Wed, 22 Jan 2014 17:37:08 GMT
Thanks for your answer, but I do not see how this could work in my case. 
My problem is that XtreemFS uses Threads (RPC clients) for method calls 
and therefore I have to stop this threads by myself to invoke the 
shutdown hooks. I think replacing one shutdown hook with another does 
not work in my case.My idea was to determine if the file system can be 
closed, because the mapper/reducer is done instead of waiting for the 
shutdown hook. So is this possible? E.g. getting the status of the task 
from somewhere an checking if it is finished?

Am 22.01.2014 01:38, schrieb Oleg Zhurakousky:
> I am not sure either, you have to ask Hadoop guys, but it was giving 
> me a hard time so I found a way around it.
> On Tue, Jan 21, 2014 at 6:05 PM, Jay Vyas <jayunit100@gmail.com 
> <mailto:jayunit100@gmail.com>> wrote:
>     I guess im not sure what the ShutdownHook actually is there
>     for.... Thats the real question im asking .
>     On Tue, Jan 21, 2014 at 5:58 PM, Oleg Zhurakousky
>     <oleg.zhurakousky@gmail.com <mailto:oleg.zhurakousky@gmail.com>>
>     wrote:
>         No, all I do is have my own shutdown hook in the main which
>         closes the FSDataOutputStream. Before I did that it would
>         throw an ugly exception when I hit Ctrl+C, telling me that the
>         stream is already closed, because of this shutdown hook (bad
>         design on the hadoop part), so removing it keeps it open until
>         the client's shutdown hook will close it at due time. So now
>         it shuts down gracefully and under complete control of the
>         client.
>         On Tue, Jan 21, 2014 at 12:30 PM, Jay Vyas
>         <jayunit100@gmail.com <mailto:jayunit100@gmail.com>> wrote:
>             what is happening when you remove the shutdown hook ?   
>             is that supposed to  trigger an exception -
>     -- 
>     Jay Vyas
>     http://jayunit100.blogspot.com

View raw message