hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-4504) DFSOutputStream#close doesn't always release resources (such as leases)
Date Fri, 15 Feb 2013 06:41:12 GMT
Colin Patrick McCabe created HDFS-4504:

             Summary: DFSOutputStream#close doesn't always release resources (such as leases)
                 Key: HDFS-4504
                 URL: https://issues.apache.org/jira/browse/HDFS-4504
             Project: Hadoop HDFS
          Issue Type: Bug
            Reporter: Colin Patrick McCabe

{{DFSOutputStream#close}} can throw an {{IOException}} in some cases.  One example is if there
is a pipeline error and then pipeline recovery fails.  Unfortunately, in this case, some of
the resources used by the {{DFSOutputStream}} are leaked.  One particularly important resource
is file leases.

So it's possible for a long-lived HDFS client, such as Flume, to write many blocks to a file,
but then fail to close it.  Unfortunately, the {{LeaseRenewerThread}} inside the client will
continue to renew the lease for the "undead" file.  Future attempts to close the file will
just rethrow the previous exception, and no progress can be made by the client.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message