hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jian Fang <jian.fang.subscr...@gmail.com>
Subject Re: HTTP ERROR 500 when call application master proxy URL in Hadoop 2.1.0-beta
Date Wed, 11 Sep 2013 21:59:26 GMT
Just in case someone run into the same problem and need help. One
workaround is to set "yarn.web-proxy.address" in yarn-site.xml explicitly
and start the proxy server as a separate process by running the following
command.

./yarn-daemon.sh start proxyserver


On Tue, Sep 10, 2013 at 1:14 PM, Jian Fang <jian.fang.subscribe@gmail.com>wrote:

> Ok, seems there is a jira for this issue.
>
> https://issues.apache.org/jira/browse/YARN-800
>
>
> On Mon, Sep 9, 2013 at 3:39 PM, Jian Fang <jian.fang.subscribe@gmail.com>wrote:
>
>> Hi,
>>
>> I need to use the web services in application master, for example,
>>
>>  curl
>> http://10.6.179.230:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>>
>> This used to work fine, but after I upgraded to Hadoop 2.1.0-beta, I got
>> the following errors.
>>
>> <html>
>> <head>
>> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
>> <title>Error 500 Connection refused</title>
>> </head>
>> <body><h2>HTTP ERROR 500</h2>
>> <p>Problem accessing
>> /proxy/application_1378761541170_0003/ws/v1/mapreduce/info. Reason:
>> <pre>    Connection refused</pre></p><h3>Caused
>> by:</h3><pre>java.net.ConnectException: Connection refused
>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>         at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
>>         at
>> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
>>         at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>>         at java.net.Socket.connect(Socket.java:529)
>>         at java.net.Socket.connect(Socket.java:478)
>>         at java.net.Socket.&lt;init&gt;(Socket.java:375)
>>         at java.net.Socket.&lt;init&gt;(Socket.java:249)
>>         at
>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
>>         at
>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
>>         at
>> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
>>         at
>> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
>>         at
>> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
>>         at
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
>>         at
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
>>         at
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.proxyLink(WebAppProxyServlet.java:185)
>>         at
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.doGet(WebAppProxyServlet.java:334)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>
>> Since there was no jetty logs for the calls, I used tcpdump to capture
>> the packets to see what was going on.
>>
>> Here is the TCP session:
>>
>> GET /ws/v1/mapreduce/info HTTP/1.1
>> Accept: application/json
>> User-Agent: Java/1.6.0_43
>> Cookie: proxy-user=dr.who
>> Host: 10.140.14.159:39975
>>
>> HTTP/1.1 302 Found
>> Cache-Control: no-cache
>> Expires: Mon, 09 Sep 2013 22:00:35 GMT
>> Date: Mon, 09 Sep 2013 22:00:35 GMT
>> Pragma: no-cache
>> Content-Type: text/plain; charset=utf-8
>> Location:
>> http://10.140.14.159:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>> Content-Length: 0
>> Server: Jetty(6.1.26)
>>
>> Looked like the application master did not redirect the web service call
>> correctly (10.140.14.159 is the slave node where the application master was
>> running and 9026 is my resource manager http port).
>>
>> There was not firewall between machines and it worked if I use the
>> following command to call the application master web services directly.
>>
>> curl http://10.140.14.159:39975/ws/v1/mapreduce/info
>>
>> Anything wrong here?
>>
>> Thanks,
>>
>> John
>>
>
>

Mime
View raw message