flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trevor Grant <trevor.d.gr...@gmail.com>
Subject Re: Flink WebUI on YARN behind firewall
Date Fri, 26 Aug 2016 23:11:07 GMT
Ahh, sorry I misunderstood.

Your comment provided insight for me though.

To anyone else who is having issues, maybe the following will help them.  I
was trying to deploy Flink on an IBM BigInsights Cloud cluster (disclaimer,
I work for IBM, not trying to promote a company, but they do give me free
cluster time so I tend to play there).

At anyrate- flink_example.py is a script that will do it nice and clean for
anyone on BigInsights Cloud, and may be useful for other similar cloud YARN
deployments so I'll just leave it here...
https://github.com/rawkintrevo/bluemix-extra-services

The solution in short-
1) install flink-
2) run yarn-session.sh
3) parse the stdout from above command (see python script, easy to follow
or email me if not)
4) open the ApplicationUI via the Yarn UI, go to the #/submit tab
5) in the submit tab, IF you open with a web browser (bc it relies on
javascript) it may give you a hostname:port <- save that port
6) combine the port you just got , with an IP address you got when parsing
in step 3
7) open ssh tunneling to that IP:port

Ugly, but it does work.

best

tg



Trevor Grant
Data Scientist
https://github.com/rawkintrevo
http://stackexchange.com/users/3002022/rawkintrevo
http://trevorgrant.org

*"Fortunate is he, who is able to know the causes of things."  -Virgil*


On Fri, Aug 26, 2016 at 5:54 PM, Vijay Srinivasaraghavan <
vijikarthi@yahoo.com> wrote:

> Hi Trevor,
>
> Sorry, I think I did not explain it clearly. I am not working on the issue
> that you have mentioned but I have noticed similar problem for another JIRA
> that I am currenty working (Flink Web UI basic authentication). The
> redirection from Yarn UI to Flink does not propogate the authentication
> headers. I was planning to investigate from Yarn UI as to what exactly is
> happening. Please go-ahead and create a JIRA anyway and someone will pick
> it up.
>
> Regards
> Vijay
>
>
> On Friday, August 26, 2016 1:34 PM, Trevor Grant <trevor.d.grant@gmail.com>
> wrote:
>
>
> That's awesome Vijay, thanks for the work- will definitely help (esp as
> Flink gets picked up for more cloud cluster deployments)!
>
> Do you have the JIRA so I can watch it?
>
> Trevor Grant
> Data Scientist
> https://github.com/rawkintrevo
> http://stackexchange.com/users/3002022/rawkintrevo
> http://trevorgrant.org
>
> *"Fortunate is he, who is able to know the causes of things."  -Virgil*
>
>
> On Fri, Aug 26, 2016 at 1:14 PM, Vijay Srinivasaraghavan <
> vijikarthi@yahoo.com> wrote:
>
> Hi Trevor,
>
> I am seeing similar issue for a JIRA that I am working now. I am yet to
> trace the Yarn Web UI code to find out how the "tracking URL" is being
> handled. To ublock, you could use the tracking URL (Flink UI URL) directly
> to access Flink Web UI to by-pass Yarn UI redirection. You can find the
> tracking URI in the Job Manager log file from Yarn container.
>
> Regards
> Vijay
>
>
> On Friday, August 26, 2016 10:52 AM, Trevor Grant <
> trevor.d.grant@gmail.com> wrote:
>
>
> I decided it made the most sense to open up a new thread.
>
> I am running Flink on a cluster behind a firewall.  Things seem to be
> working fine, but when I access the YARN web-ui and click on the flink
> application-ui, i get the jobmanager ui, but it is broken.
>
> It is a broken link to a flink image and
> [image: Apache Flink Dashboard]Apache Flink Dashboard
>
>    -  Overview
>    -  Running Jobs
>    -  Completed Jobs
>    -  Task Managers
>    -  Job Manager
>    -  Submit new Job
>
> but none of the links work, no images no pretty formatting.  does anyone
> have a quick idea why this would be?  I found http://mail-archives.
> apache.org/mod_mbox/flink- user/201511.mbox/%3CCAGr9p8B_
> aKhbjLqVQQUZeO_eSz6P=eWKP+ Kg1sQ65SB0npsX2Q@mail.gmail. com%3E
> <http://mail-archives.apache.org/mod_mbox/flink-user/201511.mbox/%3CCAGr9p8B_aKhbjLqVQQUZeO_eSz6P=eWKP+Kg1sQ65SB0npsX2Q@mail.gmail.com%3E>
>
> but that seems to be more related to running jobs than accessing the UI
>
> any help would be appreciated
>
> tg
>
>
> Trevor Grant
> Data Scientist
> https://github.com/rawkintrevo
> http://stackexchange.com/ users/3002022/rawkintrevo
> <http://stackexchange.com/users/3002022/rawkintrevo>
> http://trevorgrant.org
>
> *"Fortunate is he, who is able to know the causes of things."  -Virgil*
>
>
>
>
>
>
>

Mime
View raw message