flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Yao <g...@ververica.com>
Subject Re: REST rescale with Flink on YARN
Date Tue, 28 Jan 2020 10:17:03 GMT
Hi,

You can use

    yarn application -status <APPLICATION_ID>

to find the host and port that the server is listening on (AM host & RPC
Port). If you need to access that information programmatically, take a look
at
the YarnClient [1].

Best,
Gary


[1]
https://hadoop.apache.org/docs/r2.8.5/api/org/apache/hadoop/yarn/client/api/YarnClient.html

On Thu, Jan 23, 2020 at 3:21 PM Vasily Melnik <
Vasily.Melnik@glowbyteconsulting.com> wrote:

> Hi all,
> I've found some solution for this issue.
> Problem is that with YARN ApplicationMaster URL we communicate with
> JobManager via proxy which is implemented on Jetty 6 (for Hadoop 2.6).
> So to use PATCH method we need to locate original JobManager URL.
> Using /jobmanager/config API we could get only host, but web.port is
> displayed as 0 (???)
> To find actual web port, we should parse YARN logs for jobmanager, where
> we can find something like this:
>
> *INFO org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint - Rest
> endpoint listening at <host>:<port>.*
>
> Maybe  someone knows less complicated way to find actual REST URL under
> YARN?
>
>
>
>
> С уважением,
> Василий Мельник
>
>
> On Thu, 23 Jan 2020 at 15:32, Chesnay Schepler <chesnay@apache.org> wrote:
>
>> Older versions of Jetty don't support PATCH requests. You will either
>> have to update it or create a custom Flink version that uses POST for the
>> rescale operation.
>>
>> On 23/01/2020 13:23, Vasily Melnik wrote:
>>
>> Hi all.
>> I'm using Flink 1.8 on YARN with CDH 5.12
>> When i try to perform rescale request:
>>
>> curl -v -X PATCH '<my_url>/proxy/application_1576854986116_0079/jobs/11dcfc3163936fc019e049fc841b075b/rescaling?parallelism=3
<https://slack-redir.net/link?url=http%3A%2F%2Fstrm-flink-tm-dev-1.dwh.m1.tinkoff.cloud%3A8088%2Fproxy%2Fapplication_1576854986116_0079%2Fjobs%2F11dcfc3163936fc019e049fc841b075b%2Frescaling%3Fparallelism%3D3>'
>>
>> i get a mistake:
>>
>> *Method PATCH is not defined in RFC 2068 and is not supported by the
>> Servlet API *GET and POST methods work well.
>> The Server type in response is Jetty(6.1.26.cloudera.4).
>>
>> How can i deal with this situation?
>>
>> С уважением,
>> Василий Мельник
>>
>>
>>

Mime
View raw message