hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4309) Multithreaded get through the Cache FileSystem Object to lead LeaseChecker memory leak
Date Sat, 02 May 2015 07:12:06 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525109#comment-14525109
] 

Hadoop QA commented on HDFS-4309:
---------------------------------

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |  14m 29s | Pre-patch trunk compilation is healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any @author tags.
|
| {color:green}+1{color} | tests included |   0m  0s | The patch appears to include 1 new
or modified test files. |
| {color:red}-1{color} | javac |   0m 57s | The patch appears to cause the build to fail.
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | http://issues.apache.org/jira/secure/attachment/12561230/HDFS-4309.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/10709/console |


This message was automatically generated.

> Multithreaded get through the Cache FileSystem Object to lead LeaseChecker memory leak
> --------------------------------------------------------------------------------------
>
>                 Key: HDFS-4309
>                 URL: https://issues.apache.org/jira/browse/HDFS-4309
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 0.20.205.0, 0.23.1, 0.23.4, 2.0.1-alpha
>            Reporter: WenJin Ma
>              Labels: patch
>         Attachments: HDFS-4309.patch, jmap2.log
>
>   Original Estimate: 204h
>  Remaining Estimate: 204h
>
> If multiple threads concurrently execute the following methods will result in the thread
fs = createFileSystem (uri, conf) method is called.And create multiple DFSClient, start at
the same time LeaseChecker daemon thread, may not be able to use shutdownhook close it after
the process, resulting in a memory leak.
> {code}
> private FileSystem getInternal(URI uri, Configuration conf, Key key) throws IOException{
>       FileSystem fs = null;
>       synchronized (this) {
>         fs = map.get(key);
>       }
>       if (fs != null) {
>         return fs;
>       }
>       //  this is 
>       fs = createFileSystem(uri, conf);
>       synchronized (this) {  // refetch the lock again
>         FileSystem oldfs = map.get(key);
>         if (oldfs != null) { // a file system is created while lock is releasing
>           fs.close(); // close the new file system
>           return oldfs;  // return the old file system
>         }
>         // now insert the new file system into the map
>         if (map.isEmpty() && !clientFinalizer.isAlive()) {
>           Runtime.getRuntime().addShutdownHook(clientFinalizer);
>         }
>         fs.key = key;
>         map.put(key, fs);
>         if (conf.getBoolean("fs.automatic.close", true)) {
>           toAutoClose.add(key);
>         }
>         return fs;
>       }
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message