hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4767) Network issues can cause persistent RM UI outage
Date Wed, 25 May 2016 22:20:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-4767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300984#comment-15300984
] 

Daniel Templeton commented on YARN-4767:
----------------------------------------

The full trace looks like this:

# Thread 1) Proxy receives incoming request
# Thread 1) Proxy opens connection to the NM
# Thread 1) NM sends redirect back to the proxy
# Thread 1) Proxy opens a new connection to the proxy
# Thread 2) Proxy receives incoming request
# Thread 2) Proxy opens connection to the NM
# Thread 2) NM sends redirect back to the proxy
# Thread 2) Proxy opens a new connection to the proxy

And so on.

I'm not aware of a TTL at the HTTP level.  It's technically possible to use an HTTP header
to flag that the incoming connection is a redirect rather than encoding it in the URL, but
it would be a pretty invasive change.  The code that handles the redirects is buried pretty
deep.  I think it may even be in a bundled library.  Getting to it to make it respect the
header probably isn't worth it.

> Network issues can cause persistent RM UI outage
> ------------------------------------------------
>
>                 Key: YARN-4767
>                 URL: https://issues.apache.org/jira/browse/YARN-4767
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: webapp
>    Affects Versions: 2.7.2
>            Reporter: Daniel Templeton
>            Assignee: Daniel Templeton
>            Priority: Critical
>         Attachments: YARN-4767.001.patch, YARN-4767.002.patch, YARN-4767.003.patch, YARN-4767.004.patch,
YARN-4767.005.patch, YARN-4767.006.patch
>
>
> If a network issue causes an AM web app to resolve the RM proxy's address to something
other than what's listed in the allowed proxies list, the AmIpFilter will 302 redirect the
RM proxy's request back to the RM proxy.  The RM proxy will then consume all available handler
threads connecting to itself over and over, resulting in an outage of the web UI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message