hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ChenFolin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-4309) Multithreaded get through the Cache FileSystem Object to lead LeaseChecker memory leak
Date Thu, 13 Dec 2012 09:43:23 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

ChenFolin updated HDFS-4309:
----------------------------

    Attachment: HDFS-4309.patch
    
> Multithreaded get through the Cache FileSystem Object to lead LeaseChecker memory leak
> --------------------------------------------------------------------------------------
>
>                 Key: HDFS-4309
>                 URL: https://issues.apache.org/jira/browse/HDFS-4309
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 0.20.205.0, 0.23.1, 0.23.4, 2.0.1-alpha, 2.0.2-alpha
>            Reporter: MaWenJin
>              Labels: patch
>         Attachments: HDFS-4309.patch, jmap2.log
>
>   Original Estimate: 204h
>  Remaining Estimate: 204h
>
> If multiple threads concurrently execute the following methods will result in the thread
fs = createFileSystem (uri, conf) method is called.And create multiple DFSClient, start at
the same time LeaseChecker daemon thread, may not be able to use shutdownhook close it after
the process, resulting in a memory leak.
> private FileSystem getInternal(URI uri, Configuration conf, Key key) throws IOException{
>       FileSystem fs = null;
>       synchronized (this) {
>         fs = map.get(key);
>       }
>       if (fs != null) {
>         return fs;
>       }
>       //  this is 
>       fs = createFileSystem(uri, conf);
>       synchronized (this) {  // refetch the lock again
>         FileSystem oldfs = map.get(key);
>         if (oldfs != null) { // a file system is created while lock is releasing
>           fs.close(); // close the new file system
>           return oldfs;  // return the old file system
>         }
>         // now insert the new file system into the map
>         if (map.isEmpty() && !clientFinalizer.isAlive()) {
>           Runtime.getRuntime().addShutdownHook(clientFinalizer);
>         }
>         fs.key = key;
>         map.put(key, fs);
>         if (conf.getBoolean("fs.automatic.close", true)) {
>           toAutoClose.add(key);
>         }
>         return fs;
>       }
>     }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message