hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From karthik tunga <karthik.tu...@gmail.com>
Subject FileSystem.close() using Threads !!!
Date Wed, 15 Jun 2011 15:08:27 GMT

I have 2 threads that copy file from hdfs and delete the directory
after copying the file.

In both the threads I use "FileSystem hdfs = FileSystem.get(conf);"
Once i finish copying and deleting I close the filesystem(
hdfs.close() in the finally block)

If one of threads does a FileSystem.close()(while the other thread is
still copying) the other threads stops copying and throws an error

java.io.IOException: Filesystem closed
        at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:226)
        at org.apache.hadoop.hdfs.DFSClient.access$600(DFSClient.java:67)
        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.close(DFSClient.java:1678)
        at java.io.FilterInputStream.close(FilterInputStream.java:155)
        at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:58)
        at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:85)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:209)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:142)
        at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1216)
        at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1197)

Should I NOT do FileSystem.close() in the finally block ?? How I solve
this issue ?


View raw message