spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <pwend...@gmail.com>
Subject Re: JSON Endpoints
Date Tue, 20 Aug 2013 23:44:39 GMT
Hey Gordon,

We don't have a specification of the API right now (in 0.7.X this is a
somewhat experimental feature). Each call returns a set of (k, v) pairs and
the list of keys is static, so you can just do a function call in the
browser to see what it returns and get the schema. The JSON protocol code
is here:

https://github.com/mesos/spark/blob/branch-0.7/core/src/main/scala/spark/deploy/JsonProtocol.scala

For 0.8 this will be more cleanly specified.

- Patrick


On Tue, Aug 20, 2013 at 4:25 PM, Gordon Hamilton <
gordon.hamilton@wandisco.com> wrote:

> Hi Patrick, that's great. I think I've possibly started a build from
> branch-0.7 but have been looking at branch-0.8 code. I'll try the formats
> you have described in the morning.
>
> Is there any documentation that outlines all that I could expect to see
> returned from the json calls?
>
> Thanks,
> Gordon
>
> On Tuesday, August 20, 2013, Patrick Wendell wrote:
>
> > In 0.7.X these are actually of the format /app?format=json. In 0.8.X they
> > will be the format you described here "/app/json". Does that work for
> you?
> >
> > - Patrick
> >
> >
> > On Tue, Aug 20, 2013 at 8:53 AM, Gordon Hamilton <
> > gordon.hamilton@wandisco.com <javascript:;>> wrote:
> >
> > > Hi guys,
> > >
> > > I'm new to Spark, so apologies if I've missed something obvious.
> > >
> > > I'm investigating the possibility of using Spark from an external UI,
> and
> > > see that Spark 0.7.0 introduced JSON endpoints into the UI which allows
> > > Spark's state to be queried.  I see in MasterWebUI.scala that "/json"
> and
> > > "/app/json" map to renderJson functions, but when I hit the url's from
> a
> > > browser, I see "No service available for [/json]" and "No service
> > available
> > > for [/app/json]" respectively.
> > >
> > > Is there any documentation on how the end points work, or could someone
> > > point me in the right direction as to where I can find more
> information?
> > >
> > > Many thanks in advance,
> > > Gordon
> > >
> > >
> > > --
> > > *Register now for* Subversion & Git Live October 2013<
> > > http://www.wandisco.com/subversion-live-2013>
> > > * **in** **Boston / San Francisco /** London*
> > >
> > > THIS MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL, PROPRIETARY, AND MAY
> > BE
> > > PRIVILEGED.  If this message was misdirected, WANdisco, Inc. and its
> > > subsidiaries, ("WANdisco") does not waive any confidentiality or
> > privilege.
> > >  If you are not the intended recipient, please notify us immediately
> and
> > > destroy the message without disclosing its contents to anyone.  Any
> > > distribution, use or copying of this e-mail or the information it
> > contains
> > > by other than an intended recipient is unauthorized.  The views and
> > > opinions expressed in this e-mail message are the author's own and may
> > not
> > > reflect the views and opinions of WANdisco, unless the author is
> > authorized
> > > by WANdisco to express such views or opinions on its behalf.  All email
> > > sent to or from this address is subject to electronic storage and
> review
> > by
> > > WANdisco.  Although WANdisco operates anti-virus programs, it does not
> > > accept responsibility for any damage whatsoever caused by viruses being
> > > passed.
> > >
> > >
> >
>
>
> --
> Gordon Hamilton I *Hadoop Development Team*
>
> WANdisco // *Non-Stop Data*
>
> e. gordon.hamilton@wandisco.com
>
> --
> *Register now for* Subversion & Git Live October 2013<
> http://www.wandisco.com/subversion-live-2013>
> * **in** **Boston / San Francisco /** London*
>
> THIS MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL, PROPRIETARY, AND MAY BE
> PRIVILEGED.  If this message was misdirected, WANdisco, Inc. and its
> subsidiaries, ("WANdisco") does not waive any confidentiality or privilege.
>  If you are not the intended recipient, please notify us immediately and
> destroy the message without disclosing its contents to anyone.  Any
> distribution, use or copying of this e-mail or the information it contains
> by other than an intended recipient is unauthorized.  The views and
> opinions expressed in this e-mail message are the author's own and may not
> reflect the views and opinions of WANdisco, unless the author is authorized
> by WANdisco to express such views or opinions on its behalf.  All email
> sent to or from this address is subject to electronic storage and review by
> WANdisco.  Although WANdisco operates anti-virus programs, it does not
> accept responsibility for any damage whatsoever caused by viruses being
> passed.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message