flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: 1.1.1: JobManager config endpoint no longer supplies port
Date Fri, 19 Aug 2016 14:48:32 GMT
Hi Shannon,

the problem is that YARNs proxy only allows GET HTTP requests, but for
uploading files, a different request type is needed.

I've filed a JIRA for the problem you've reported:
https://issues.apache.org/jira/browse/FLINK-4432

Regards,
Robert

On Mon, Aug 15, 2016 at 6:03 PM, Shannon Carey <scarey@expedia.com> wrote:

> Thanks Ufuk. For now, we will use the Yarn AM proxy.
>
> About uploading JARs: the JobManager UI that is exposed via the Yarn AM
> proxy does not allow manually uploading Flink job jars for execution on the
> cluster (look for "Yarn's AM proxy doesn't allow file uploads." in the
> code). As I understand it, this is due choices Yarn has made about security.
>
>
>
> On 8/15/16, 9:25 AM, "Ufuk Celebi" <uce@apache.org> wrote:
>
> >I've verified this. I think this has likely accidentally changed with
> >the refactoring of the YARN setup for Flink 1.1. We probably wrote the
> >web monitor port explicitly to the config in 1.0 whereas we don't do
> >it in 1.1 anymore. I think this should be addressed with the next
> >bugfix release 1.1.2.
> >
> >What comes to mind to find out the port is to parse the logs, but I
> >don't think that's feasible, is it? Would also be interested what you
> >are referring to with the JARs.
> >
> >On Mon, Aug 15, 2016 at 4:04 PM, Ufuk Celebi <uce@apache.org> wrote:
> >> Hey Shannon! I just took a look at the code and it looks like the
> >> Flink REST handler for the config did _not_ change since last year. It
> >> could be that somehow the config is loaded differently. Can you verify
> >> that using the same config with Flink 1.0 and Flink 1.1 the port is
> >> printed correctly and as 0? I will look into it as well.
> >>
> >> The JAR uploading issue is a separate one, correct?
> >>
> >> On Fri, Aug 12, 2016 at 6:03 PM, Shannon Carey <scarey@expedia.com>
> wrote:
> >>> It appears that when running Flink 1.1.1 on Yarn, my previous method of
> >>> making a request to the yarn AM proxy on the master node at
> >>> http://{master_node}:20888/proxy/{app_id}/jobmanager/config doesn't
> work the
> >>> same as it did.
> >>>
> >>> Previously, the returned JSON value would include an accurate value for
> >>> "jobmanager.web.port". Now, however, it appears to return a zero.
> >>>
> >>> As a result, I cannot expose the JobManager UI itself. I can use the
> Yarn
> >>> proxy but it does not support the useful feature of uploading JARs.
> >>>
> >>> Is there another way of determining which port the JobManager is
> running on?
> >>> Thanks!
> >>>
> >>> -Shannon
> >
>

Mime
View raw message