flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: REST API "broken" on YARN because POST is not allowed via YARN proxy
Date Wed, 04 Apr 2018 14:31:58 GMT
Hi Juho,

Thanks for raising this point!

I'll add Chesnay and Till to the thread who contributed to the REST API.

Best, Fabian

2018-04-04 15:02 GMT+02:00 Juho Autio <juho.autio@rovio.com>:

> I just learned that Flink savepoints API was refactored to require using
> HTTP POST.
>
> That's fine otherwise, but makes life harder when Flink is run on top of
> YARN.
>
> I've added example calls below to show how POST is declined by
> the hadoop-yarn-server-web-proxy*, which only supports GET and PUT.
>
> Can you think of any solution to this? If I would be able to determine the
> actual host & port for Flink UI, I could use that instead of the proxy
> address. But that would probably require opening at least one more port, so
> it's not the optimal solution either. Ideally I would have Flink REST API
> completely accessible with GET and PUT methods.
>
> To me it seems like AWS EMR will also hit this issue as soon as they start
> supporting Flink 1.5, because they seem to run Flink as a YARN app.
>
>
> *) https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-
> project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-
> server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/
> WebAppProxyServlet.java#L296-L306
>
>
> $ http POST http://10.0.10.71:20888/proxy/application_1522844153347_
> 0001/jobs/652e207f8578574d4a322e23d4f8b908/checkpoints
>
> HTTP/1.1 405 HTTP method POST is not supported by this URL
> Cache-Control: must-revalidate,no-cache,no-store
> Content-Length: 1523
> Content-Type: text/html; charset=iso-8859-1
> Date: Wed, 04 Apr 2018 12:48:05 GMT
> Date: Wed, 04 Apr 2018 12:48:05 GMT
> Pragma: no-cache
> Pragma: no-cache
>
> <html>
> <head>
> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
> <title>Error 405 HTTP method POST is not supported by this URL</title>
> </head>
> <body><h2>HTTP ERROR 405</h2>
> <p>Problem accessing /proxy/application_1522844153347_0001/jobs/
> 652e207f8578574d4a322e23d4f8b908/checkpoints. Reason:
> <pre>    HTTP method POST is not supported by this URL</pre></p><hr
> /><i><small>Powered by Jetty://</small></i><br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
> <br/>
>
> </body>
> </html>
>
>
> $ http PUT http://10.0.10.71:20888/proxy/application_1522844153347_
> 0001/jobs/652e207f8578574d4a322e23d4f8b908/checkpoints
>
> HTTP/1.1 404 Not Found
> Access-Control-Allow-Origin: *
> Cache-Control: no-cache
> Content-Length: 25
> Content-Type: application/json; charset=UTF-8
> Date: Wed, 04 Apr 2018 12:48:09 GMT
> Date: Wed, 04 Apr 2018 12:48:09 GMT
> Expires: Wed, 04 Apr 2018 12:48:09 GMT
> Expires: Wed, 04 Apr 2018 12:48:09 GMT
> Pragma: no-cache
> Pragma: no-cache
>
> {
>     "errors": [
>         "Not found."
>     ]
> }
>
> ^ expected to get this from flink because there's no PUT /checkpoints.
>

Mime
View raw message