mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Anderegg <timothy.ander...@gmail.com>
Subject Re: Catching the webui up to features
Date Wed, 18 Oct 2017 17:36:35 GMT
We have a similar scenario using an actual proxy (Apache in this case) in
front of a firewalled VLAN, that also does Kerberos authentication against
the endpoints.  This currently works fine for external API requests but the
WebUI is only partially functional, since I haven't looked into the newer
Mesos features to support proxying that Ben mentioned.  I will try to
implement those and see if I can get it all working as it should.  Is there
documentation on those features?  If not I will figure it out and submit
some :)

YARN handles this differently (like Aaron thought Mesos would), such that
the web UI has its own proxy that allows access to some of the services
running on it without having to do more complex standalone proxy
configurations or service discovery, but implementing that for Mesos would
certainly increase the complexity of Mesos' web UI.  This does enable some
nice features such as being able to access the Spark UI automatically via
the Yarn UI, without having to worry about where the driver is running or
what port it is on.

- Tim

On Wed, Oct 18, 2017 at 1:11 PM Aaron Wood <aaronjwood@gmail.com> wrote:

> In this case localhost is my host (OSX) and u64 is my guest (Ubuntu VM).
> What I was doing was viewing the Mesos UI from my host by making use of
> port mapping in my VM. Port 10050 on my host maps to 5050 inside my VM. So
> in a way this is like viewing the UI with a proxy in the middle. What I was
> expecting was that I could view the UI served from the master and not have
> to worry about reaching the agents directly for their information. I'm not
> using DC/OS at all here, just my own builds of Apache Mesos.
>
> I was thinking that if the master was able to handle the communication with
> the agents (and other masters if the leader changes) that people would be
> able to use the UI in situations like mine, or more realistically, in
> situations where you have some sort of cluster that you need to tunnel into
> in order to reach Mesos. For example, Mesos is not exposed outside of our
> cluster directly since the cluster has its own private internal network.
> You have to tunnel in using our defined node(s) that are on both the
> external and internal network.
>
> I hope that helps explain the scenario!
>
> Thanks,
> Aaron
>
> On Tue, Oct 17, 2017 at 3:05 PM, Benjamin Mahler <bmahler@apache.org>
> wrote:
>
> > What does localhost vs u64 represent for you there in terms of your
> actual
> > cluster?
> >
> > Are you expecting to be able to publicly access the masters and agents?
> Or
> > are you running a proxy (like DC/OS's adminrouter)?
> > We recently made some changes that makes it easier to patch the UI to
> work
> > with a proxy.
> >
> > Would be great to understand how you'd like it to work.
> >
> > Ben
> >
> > On Tue, Sep 26, 2017 at 9:15 AM, Aaron Wood <aaronjwood@gmail.com>
> wrote:
> >
> > > You're right about the slaves section in 0.28.x. There are no error
> > > messages or modals that pop up but looking at the requests after you
> > click
> > > on a slave in the slaves section shows these obvious failures (u64 is
> the
> > > hostname of my VM which won't be resolved from the host):
> > >
> > > Request URL:
> > > http://u64:5051/slave(1)/state?jsonp=angular.callbacks._6
> > > Request URL:
> > > http://u64:5051/metrics/snapshot?jsonp=angular.callbacks._7
> > > Request URL:
> > > http://u64:5051/monitor/statistics?jsonp=angular.callbacks._8
> > >
> > > What does work in 0.28.x (and doesn't in 1.0.x) is the main page which
> > has
> > > AJAX calls on an interval. These are successfully completed every 10
> > > seconds or so:
> > >
> > > Request URL:
> > > http://localhost:10050/master/state
> > > Request URL:
> > > http://localhost:10050/metrics/snapshot
> > >
> > > In the newer versions of Mesos these requests are a bit different and
> > fail:
> > >
> > > Request URL:
> > > http://u64:5050/master/state?jsonp=angular.callbacks._2
> > > Request URL:
> > > http://u64:5050/metrics/snapshot?jsonp=angular.callbacks._3
> > >
> > > This is what causes the error modal to pop up on every interval tick of
> > > these AJAX requests.
> > >
> > > Thanks,
> > > Aaron
> > >
> > > On Mon, Sep 25, 2017 at 7:47 PM, Benjamin Mahler <bmahler@apache.org>
> > > wrote:
> > >
> > > > 0.28.x UI is working for you even when you click into an agent?
> > > >
> > > > Can you show us which requests are being made in the old UI vs the
> new
> > UI
> > > > and which ones do not work for you? (use chrome's built-in debugging
> > > tools)
> > > >
> > > > On Mon, Sep 25, 2017 at 3:38 PM, Aaron Wood <aaronjwood@gmail.com>
> > > wrote:
> > > >
> > > > > I just built Mesos 0.28.x and tried out this scenario. It seems
> that
> > > this
> > > > > issue presents itself only in 1.0.x and up. I also found out that
> > DCOS
> > > > > hides this issue if someone uses
> https://github.com/dcos/adminrouter
> > > > >
> > > > > I have not dug through the 0.28.x code enough to pinpoint the exact
> > > area
> > > > > for where the logic is different. Maybe someone that has worked on
> > the
> > > UI
> > > > > in the past can shed more light on this...?
> > > > >
> > > > > On Wed, Sep 13, 2017 at 8:03 PM, Benjamin Mahler <
> bmahler@apache.org
> > >
> > > > > wrote:
> > > > >
> > > > > > +haosdent, vinod
> > > > > >
> > > > > > Blaming the lines you pointed to gives me:
> > > > > >
> > > > > > https://github.com/apache/mesos/commit/
> > > b2fc58883e2cd0ca144fd1b0e10cad
> > > > > > 4235a50223
> > > > > >
> > > > > > I've added vinod and haosdent since they were involved in the
> > change,
> > > > it
> > > > > > looks like this makes the master talk directly to a different
> > master
> > > > > > host:port. Is this what breaks for you?
> > > > > >
> > > > > > If so, I'm not sure why pre-1.0.x would work for you if you
were
> to
> > > > click
> > > > > > into an agent, as that will also make a request directly to
an
> > agent
> > > > > > host:port.
> > > > > >
> > > > > > Ben
> > > > > >
> > > > > > On Mon, Sep 11, 2017 at 7:44 PM, Aaron Wood <
> aaronjwood@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Hey Ben,
> > > > > > >
> > > > > > > There was a change sometime after Mesos 1.0.x (I think)
that
> > > altered
> > > > > how
> > > > > > > the leader state was obtained
> > > > > > > https://github.com/apache/mesos/blob/master/src/webui/
> > > > > > > master/static/js/controllers.js#L362-L364
> > > > > > >
> > > > > > > This seems to be an intended change but this causes UI
error
> > modals
> > > > to
> > > > > > > continually pop up stating that the leader could not be
reached
> > > when
> > > > > > > accessing the UI over a tunnel. The errors also show when
you
> try
> > > to
> > > > > > access
> > > > > > > agent info among other things throughout the angular app.
> > > > > > >
> > > > > > > I'm curious, how many people using Mesos access the UI
over a
> > > tunnel?
> > > > > Is
> > > > > > > there any harm in reverting this functionality to how it
was in
> > the
> > > > > 1.0.x
> > > > > > > days? I'm sure there was some important reason as to why
this
> > > change
> > > > > was
> > > > > > > made, I'm just not aware of it. From my understanding many
of
> the
> > > ops
> > > > > > folks
> > > > > > > in our org use the UI but almost always over a tunnel.
Once
> more
> > > > groups
> > > > > > in
> > > > > > > the org upgrade to newer versions of Mesos they will be
facing
> > this
> > > > > issue
> > > > > > > almost on a daily basis.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Aaron
> > > > > > >
> > > > > > > On Mon, Sep 11, 2017 at 2:59 PM, Benjamin Mahler <
> > > bmahler@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi folks,
> > > > > > > >
> > > > > > > > Over time the webui has lagged behind for some of
the
> features
> > > that
> > > > > > have
> > > > > > > > been added. I'm currently tracking what's required
to catch
> it
> > up
> > > > > here:
> > > > > > > >
> > > > > > > > https://issues.apache.org/jira/browse/MESOS-6440
> > > > > > > >
> > > > > > > > If you know of other features that make sense to display
in
> the
> > > > > webui,
> > > > > > > feel
> > > > > > > > free to file a ticket under this epic (or link it
as related
> if
> > > it
> > > > > > falls
> > > > > > > > under a different epic) and let me know. For example,
I just
> > > filed
> > > > > > > another
> > > > > > > > one within it for displaying task health information.
> > > > > > > >
> > > > > > > > Also feel free to make contributions to the webui
even if you
> > > don't
> > > > > > feel
> > > > > > > > that you're knowledgable on the frontend side of things.
The
> > > > majority
> > > > > > of
> > > > > > > > webui changes are very easy and provide a lot of value
to
> users
> > > who
> > > > > > > > interact with it on a regular basis!
> > > > > > > >
> > > > > > > > If you'd like to contribute to the webui, there are
a lot of
> > easy
> > > > > > tickets
> > > > > > > > to get started with, here is one example that I would
be
> happy
> > to
> > > > > > assist
> > > > > > > > with: https://issues.apache.org/jira/browse/MESOS-7962
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > > Ben
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message