aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Farner <>
Subject Re: Aurora Client over HTTP/JSON
Date Wed, 23 Apr 2014 17:28:33 GMT
Firstly, i think it's _awesome_ that you're hacking with our code!  Thanks
for sharing a pointer to your fork!

How would you feel about integrating this kind of thing into the scheduler
rather than a separate webapp?  I'd be happy to help acquaint you with the
scheduler code if that presents a hurdle.


On Tue, Apr 22, 2014 at 2:04 PM, Madan, Smarth <> wrote:

>  Hi Bill,
>  I created a REST client for scheduling jobs on Aurora. It uses the
> Thrift generated java and connects to the Aurora Scheduler server skipping
> the Python client. I also built a simple UI on top of it. Currently I am
> testing the UI. My code is in my repository forked from the original Aurura
> repo : . I think it
> would be a useful tool to schedule jobs using a simple UI and let people
> try it out easily.
> I could raise a pull request once I am done testing for the same. Please
> let me know your suggestions.
>  Thanks
> Smarth Madan
>   From: Bill Farner <>
> Date: Thursday, March 27, 2014 at 12:34 PM
> To: "" <>
> Cc: "Madan, Smarth" <>
> Subject: Re: Aurora Client over HTTP/JSON
>    Hi Smarth!
> This is out of scope of AURORA-20.  That ticket tracks creating a guide for new developers
on the client.  However, i'm not opposed to the feature you propose.
> The client currently speaks HTTP/JSON.  If you invoke ./gradlew run, you might notice
these API calls when you view some of the pages on the scheduler web interface.  Chrome developer
tools makes it easy to convert these calls into curl commands.  Here's one stripped down:
> $ curl 'http://localhost:8081/api' --data-binary '[1,"getRoleSummary",1,0,{}]'
> [1,"getRoleSummary",2,0,{"0":{"rec":{"1":{"i32":1},"3":{"rec":{"17":{"rec":{"1":{"set":["rec",1,{"1":{"str":"mesos"},"2":{"i32":40},"3":{"i32":20}}]}}}}},"4":{"rec":{"1":{"i32":3}}},"5":{"rec":{"1":{"str":"local"},"2":{"i32":3}}}}}}]
> This is invoking methods on the thrift API [1], and results are encoded using TJSONProtocol
(configured in SchedulerAPIServlet [2]).  You'll notice from the output above that TJSONProtocol
encodes fields with their numeric field IDs, as specified in api.thrift.  This is a fundamental
characteristic of IDLs like thrift and protobuf - field IDs rather than names are transmitted
over the wire.  Obviously, this is somewhat at odds with the traditional HTTP/JSON approach.
> I see three obvious paths forward, some of which might not suit your needs:
> - use a part of our client as a python API from your tooling
> - use thrift-compiled code in your tooling
> - augment the scheduler to have separate endpoints that expose fields by name rather
than numeric ID.  there may be prior art for this, but otherwise would look something like
SchedulerAPIServlet along with a different TProtocolFactory [3].
>  Hope that helps!  Cheers!
>  -=Bill
>  [1]
> [2]
> [3]

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message