hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-7317) RPC.stopProxy doesn't actually close proxy
Date Wed, 09 Nov 2011 08:25:51 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vinod Kumar Vavilapalli updated HADOOP-7317:
--------------------------------------------

    Fix Version/s:     (was: 0.22.0)
                   0.23.1

I got bitten by this at MAPREDUCE-3333. MR AM creates lots of connections to NodeManagers
to launch containers and on a big enough cluster, MR AM is hitting the #processes limits set
on nodes (2048). I am working around temporarily by setting the RPC client maxIdleTime to
zero.

Any ideas on how we can fix this?
                
> RPC.stopProxy doesn't actually close proxy
> ------------------------------------------
>
>                 Key: HADOOP-7317
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7317
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: ipc
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>             Fix For: 0.23.1
>
>
> Discovered while investigating HDFS-1965, it turns out that the reference-counting done
in WritableRpcEngine.ClientCache doesn't map one-to-one with open TCP connections. This means
that it's easy to accidentally leave TCP connections open longer than expected so long as
the client has any other connections open at all.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message