hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4824) FileInputStreamCache.close leaves dangling reference to FileInputStreamCache.cacheCleaner
Date Thu, 16 May 2013 21:23:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660018#comment-13660018

Colin Patrick McCabe commented on HDFS-4824:

OK.  I think I can see your logic.  The references I was talking about were references from
the DFSInputStream to the DFSClient.  But neither of those are GC roots, whereas {{FileInputStreamCache#executor}}
certainly is.  So there may be some value in breaking the connection from the GC root to the
{{FileInputStreamCache}}, which is after all a per-{{DFSInputStream}} object.

bq. With your fix in place, if you leak a DFSInputStream without closing, does it eventually
get GCed in a full GC? Or would it stay around forever? I don't think there are other hanging
references to DFSInputStreams, but maybe I'm wrong.

I didn't see any in my cursory inspection.  It's pretty difficult to tell without dynamic
instrumentation, though.
> FileInputStreamCache.close leaves dangling reference to FileInputStreamCache.cacheCleaner
> -----------------------------------------------------------------------------------------
>                 Key: HDFS-4824
>                 URL: https://issues.apache.org/jira/browse/HDFS-4824
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 2.0.4-alpha
>            Reporter: Henry Robinson
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-4824.001.patch
> {{FileInputStreamCache}} leaves around a reference to its {{cacheCleaner}} after {{close()}}.
> The {{cacheCleaner}} is created like this:
> {code}
> if (cacheCleaner == null) {
>           cacheCleaner = new CacheCleaner();
>           executor.scheduleAtFixedRate(cacheCleaner, expiryTimeMs, expiryTimeMs,
>               TimeUnit.MILLISECONDS);
>         }
> {code}
> and supposedly removed like this:
> {code}
> if (cacheCleaner != null) {
>   executor.remove(cacheCleaner);
> }
> {code}
> However, {{ScheduledThreadPoolExecutor.remove}} returns a success boolean which should
be checked. And I _think_ from a quick read of that class that the return value of {{scheduleAtFixedRate}}
should be used as the argument to {{remove}}. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message