hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <oleg.zhurakou...@gmail.com>
Subject Re: Shutdown hook for FileSystems
Date Tue, 21 Jan 2014 16:59:20 GMT
Had the same problem recently
Here is how you can get around it.

I am using org.springframework.util.ReflectionUtils here just for
convenience (to traverse class hierarchy if necessary), but you can just
use raw reflection.

try {

 Field clientFinalizerField = ReflectionUtils.findField(fileSystem.getClass(),


 Thread clientFinalizer = (Thread) clientFinalizerField.get(null);


 } catch (Exception e) {

 // handle


On Tue, Jan 21, 2014 at 5:36 AM, Lukas Kairies <
lukas.xtreemfs@googlemail.com> wrote:

> Hey,
> I use Hadoop with XtreemFS (with a corresponding FileSystem
> implementation). The XtreemFS client uses several non-deamon Threads eg.
> for communication. Therefore the shutdown hooks do not start after a
> mapper/reducer is finished and the Child processes do not terminate.
> My question:
> Is there any way to determine if the mapper/reducer child processes do not
> need the file system any more?
> Best regards,
> Lukas K

View raw message