Mailing-List: contact dev-help@aurora.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@aurora.apache.org
MIME-Version: 1.0
In-Reply-To: <102226956.3142559.1437982491897.JavaMail.yahoo@mail.yahoo.com>
References: 
 <CAGRA8uNPsaVJud1HObGFfq9jN43TxJK3CUe-SzfWwP8by_gn+g@mail.gmail.com>
	<102226956.3142559.1437982491897.JavaMail.yahoo@mail.yahoo.com>
Date: Thu, 30 Jul 2015 08:31:05 -0700
Message-ID: 
 <CAGRA8uNeJvOicnC+8NBB1s0DWK1DkCKZQwnqxmUc1pwqy9hbCg@mail.gmail.com>
Subject: Re: Aurora-1288 custom executor support in client
From: Bill Farner <wfarner@apache.org>
To: "dev@aurora.apache.org" <dev@aurora.apache.org>,
 meghdoot bhattacharya <meghdoot_b@yahoo.com>
Content-Type: multipart/alternative; boundary=001a11c2ac5e457de4051c1964a6

--001a11c2ac5e457de4051c1964a6
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

>
> Is the REST API interface planned to be be identical/close to the thrift
> api interface?


The REST API is likely to be a replacement for the thrift API, so it will
indeed require feature parity to reach that point.

And for example, is AcquireLock scheduler method still used?


AcquireLock is only used for client-side updates, and will likely be phased
out with them.

On the CreateJob, is a lock object required?


No.

It would be good to document these, so a generic client can be written.
> Maybe that is what this ticket should do.


There's some documentation in api.thrift [1], which is in turn rendered
into HTML and served by the scheduler.  However, expanding on the
documentation would be great.

[1]
https://github.com/apache/aurora/blob/master/api/src/main/thrift/org/apache=
/aurora/gen/api.thrift


-=3DBill

On Mon, Jul 27, 2015 at 12:34 AM, meghdoot bhattacharya <
meghdoot_b@yahoo.com.invalid> wrote:

> Is the REST API interface planned to be be identical/close to the thrift
> api interface?
> And if we leave the client dsl modifications for custom executors, for no=
w
> we can build a generic client to use the thrift api directly for supporti=
ng
> custom executors. Cursory look of the client api code looks like the
> scheduler proxy object is calling the scheduler directly but would be nic=
e
> to have the thick client logic completely removed. The old style client
> orchestrated updater code is very much present. And for example, is
> AcquireLock scheduler method still used? Looks like new update method doe=
s
> not require it but the old update method is the only one that needed it.
> But if latter is deprecated, why not remove the scheduler method? On the
> CreateJob, is a lock object required? It does not seem necessary. It woul=
d
> be good to document these, so a generic client can be written. Maybe that
> is what this ticket should do.
> Thx
>       From: Bill Farner <wfarner@apache.org>
>  To: "dev@aurora.apache.org" <dev@aurora.apache.org>; meghdoot
> bhattacharya <meghdoot_b@yahoo.com>
>  Sent: Tuesday, July 21, 2015 9:53 PM
>  Subject: Re: Aurora-1288 custom executor support in client
>
> One of the biggest challenges i foresee if we embark on making the client
> support custom executors is schema handling.  By design, pystachio define=
s
> schemas that configurations must match, which will likely make it difficu=
lt
> to generalize to support swappable/arbitrary executors.  I am not familia=
r
> enough with pystachio to articulate how [in]feasible this is, so hopefull=
y
> someone else can expound on that.
>
> While i think this is possible, i wouldn't be disappointed if this use ca=
se
> were used to apply pressure to getting a REST API in place, as i believe
> that's one of the few major hurdles left necessitating a CLI.
>
>
> -=3DBill
>
>
>
> On Tue, Jul 21, 2015 at 2:42 PM, meghdoot bhattacharya <
> meghdoot_b@yahoo.com.invalid> wrote:
>
> > https://issues.apache.org/jira/browse/AURORA-1288
> > As the server side changes are getting wrapped up, Bill and I were
> > discussing offline around custom executor support in aurora client, one
> of
> > the sub tasks in the ticket. So, we are bringing the discussion to the
> > community to get feedback from you all.
> > Meghdoot>Theidea should be to support the current DSL (to not break
> > current integrations)but also introduce a new way of defining things
> where
> > even thermos or anycustom executor can have its own stuff defined in a
> > generic way eventuallymarshalled into =E2=80=9Cdata=E2=80=9D blob suppo=
rted as Brian had
> > commented in the ticket
> >
> > Bill> Just so i follow - what drives the requirement ofdoing this in th=
e
> > existing client?  Our current direction is to make theclient thinner ov=
er
> > time, so hypothetically the surface area for a custom clientis getting
> > pretty small.  I'd also like to invest in a REST API quitesoon, which
> > reduces the technology coupling as well.  Do these detailschange the
> > equation for you?
> >
> >
> > Meghdoot>Generallyhaving a cli is powerful. Though we can rely on thrif=
t
> > api=E2=80=99s and future restapi=E2=80=99s directly to plug in with our=
 web systems,
> having
> > a cli support is apowerful functionality to have especially if we can
> make
> > it generic to dealwith custom executors. And we are thinking of upgradi=
ng
> > the aurora client andnot have some other cli.
> >
> > Inthe end any client should just send the job config and executor data
> > buried inthe current field, but having a DSL where you can declare thin=
gs
> > can still bebeneficial.
> >
> >  Butif we really are moving in a direction that the existing cli gets
> used
> > mostlyfor admin usage and we retire the DSL usage, then there is no poi=
nt
> > investingon it. Otherwise the question remains why DSL support only for
> > thermos.
> > Otherpoint to add if we really want to keep the DSL only for thermos,
> then
> > to havecustom executors thrive through clients of their own, we need a
> > detailed writeup on how that will work. Meaning to create or update or
> some
> > other job relatedoperations what are the sequence of API=E2=80=99s to c=
all (for
> > example grab lock api,call main api, call release api), if behind the
> > scenes the existing aurora client isdoing it today. Otherwisecustom
> > executors will struggle in the absence of support from aurora client
> andan
> > integration guide.
> >
> >
> >
> >
> >
> >
> >
> >
>
>
>

--001a11c2ac5e457de4051c1964a6--