hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4754) Too many connection opened to TimelineServer while publishing entities
Date Thu, 03 Mar 2016 08:58:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15177506#comment-15177506
] 

Rohith Sharma K S commented on YARN-4754:
-----------------------------------------

bq. I still see 2 places where we are not closing ClientResponse, when we call putDomain and
in doPosting if response is not 200 OK.
It looks to be this is the case. After RM recovery completes, timeline entities are published
in background. During this span of time, if there timeline sever is restarted or down for
sometime, it is able to see many connections are kept CLOSE_WAIT state.

> Too many connection opened to TimelineServer while publishing entities
> ----------------------------------------------------------------------
>
>                 Key: YARN-4754
>                 URL: https://issues.apache.org/jira/browse/YARN-4754
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Rohith Sharma K S
>            Priority: Critical
>         Attachments: ConnectionLeak.rar
>
>
> It is observed that there are too many connections are kept opened to TimelineServer
while publishing entities via SystemMetricsPublisher. This cause sometimes resource shortage
for other process or RM itself
> {noformat}
> tcp        0      0 10.18.99.110:3999       10.18.214.60:59265      ESTABLISHED 115302/java
        
> tcp        0      0 10.18.99.110:25001      :::*                    LISTEN      115302/java
        
> tcp        0      0 10.18.99.110:25002      :::*                    LISTEN      115302/java
        
> tcp        0      0 10.18.99.110:25003      :::*                    LISTEN      115302/java
        
> tcp        0      0 10.18.99.110:25004      :::*                    LISTEN      115302/java
        
> tcp        0      0 10.18.99.110:25005      :::*                    LISTEN      115302/java
        
> tcp        1      0 10.18.99.110:48866      10.18.99.110:8188       CLOSE_WAIT  115302/java
        
> tcp        1      0 10.18.99.110:48137      10.18.99.110:8188       CLOSE_WAIT  115302/java
        
> tcp        1      0 10.18.99.110:47553      10.18.99.110:8188       CLOSE_WAIT  115302/java
        
> tcp        1      0 10.18.99.110:48424      10.18.99.110:8188       CLOSE_WAIT  115302/java
        
> tcp        1      0 10.18.99.110:48139      10.18.99.110:8188       CLOSE_WAIT  115302/java
        
> tcp        1      0 10.18.99.110:48096      10.18.99.110:8188       CLOSE_WAIT  115302/java
        
> tcp        1      0 10.18.99.110:47558      10.18.99.110:8188       CLOSE_WAIT  115302/java
        
> tcp        1      0 10.18.99.110:49270      10.18.99.110:8188       CLOSE_WAIT  115302/java
        
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message