Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Thu, 10 Jul 2014 17:52:06 +0000 (UTC)
From: "Chris Nauroth (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.12725823.1404760904530.17368.1405014726815@arcas>
In-Reply-To: <JIRA.12725823.1404760904530@arcas>
References: <JIRA.12725823.1404760904530@arcas>
Subject: [jira] [Commented] (HADOOP-10792) Add FileSystem#closeIfNotReferred
 method
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14057736#comment-14057736 ] 

Chris Nauroth commented on HADOOP-10792:
----------------------------------------

bq. If I use FileSystem, using newInstance method and not invoke FileSystem#close makes sense right?

This might not be a good idea depending on exactly how you use the {{FileSystem}}.  For example:

* If you rely on delete-on-exit (see method {{FileSystem#deleteOnExit}}), then never calling {{FileSystem#close}} means those files might not get deleted in a timely fashion.  They'd still get deleted on JVM process shutdown thanks to a shutdown hook, but that might not clean up the files quickly enough if you have a long-running process.
* If this is HDFS, then closing the {{DistributedFileSystem}} instance also implicitly closes any files that are still held open for write.  If your application was dependent on this behavior, then skipping the close might cause problems.
* If this is a custom {{FileSystem}}, then the subclass implementors could have overridden {{close}} and coded any arbitrary cleanup logic that they saw necessary.  It's hard to predict what would happen if you refuse to close when you consider custom implementations.

In general, I prefer Steve's earlier recommendation.

> Add FileSystem#closeIfNotReferred method
> ----------------------------------------
>
>                 Key: HADOOP-10792
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10792
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs
>            Reporter: Kousuke Saruta
>
> FileSystem#close closes FileSystem even if the same instance of FileSystem is referred by someone.
> For instance, a library using FileSystem calls FileSystem.get, and a program using the library calls FileSystem.get, both of instances of FileSystem is same. 
> When the library and the program is implemented as different threads and one calls FileSystem.close, another fails most of operations of FileSystem.
> So, we need the method like cloesIfNotReferred, which closes FileSystem only if a instance of FileSystem is not referred by anyone.


--
This message was sent by Atlassian JIRA
(v6.2#6252)