hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From M G <mgbuyst...@gmail.com>
Subject Fwd: FileSystem object's close method is not called. hadoop 2.7.2
Date Wed, 13 Jul 2016 14:23:03 GMT
Hi, I have implemented an HC file system by extending the
org.apache.hadoop.fs.FileSystem class.
Things seem to work fine, except the fact that when I run mapreduce jobs
with yarn (simple DFSIO tests) I keep noticing that my Filesystem's "close"
method is not called.
That is, I see in logs that my file system object is instantiated seven
times during the run of the MRAppMaster process, but the close method was
never closed for any instantiation.
I know that there is a configuration key fs.automatic.close that is checked
on MRappMaster.java - and I make sure that this key is true.
I am using hadoop 2.7.2
Other processes that instantiate my FS class (node manager for example) are
calling the close method, I saw the problem only with MRAppMaster's process.
The DFSIO test succeeds, but this issue can lead to some resources leakage
so it's important that the close method is called.


View raw message