hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3359) DFSClient.close should close cached sockets
Date Mon, 27 Aug 2012 23:29:07 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442804#comment-13442804
] 

Tsz Wo (Nicholas), SZE commented on HDFS-3359:
----------------------------------------------

> Temp files are removed either by explicit close, or via an implicit close issued by the
shutdown hooks. How can a long running client force the removal the temp files prior to exit?

Sorry that I forgot to answer this question earlier.  I think the deleteOnExit feature is
for deleting files on exit, especially for unexpected exit, but not deleting the files "some
time later".  If a long running client wants to delete its temp files some time later, it
probably should add the temp files to a list and then use the list to delete them later.
                
> DFSClient.close should close cached sockets
> -------------------------------------------
>
>                 Key: HDFS-3359
>                 URL: https://issues.apache.org/jira/browse/HDFS-3359
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>    Affects Versions: 0.22.0, 2.0.0-alpha
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Critical
>             Fix For: 0.23.3, 2.0.0-alpha
>
>         Attachments: hdfs-3359-branch-0.23.txt, hdfs-3359.txt, hdfs-3359.txt
>
>
> Some applications like the TT/JT (pre-2.0) and probably the RM/NM cycle through DistributedFileSystem
objects reasonably frequently. So long as they call close() it isn't a big problem, except
that currently DFSClient.close() doesn't explicitly close the SocketCache. So unless a full
GC runs (causing the references to get finalized), many SocketCaches can get orphaned, each
with many open sockets inside. We should fix the close() function to close all cached sockets.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message