Mailing-List: contact dev-help@aurora.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@aurora.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAJGK=o6mHmMnRi9RdkBrbxGmjsSsGG7BbBr6U7M-xTXCaiyL2A@mail.gmail.com>
References: 
 <CAJGK=o4r3BVb0ij5gBK4DR+hYskq+6Y4OfBLWfDo9sUFysDfDA@mail.gmail.com>
	<CAOTkfX5S4nCVReP5CWw95FpAsQriFskrm+nPd6SDSOduN3=hRw@mail.gmail.com>
	<CAGRA8uM24NEoKLU8w4J+gO-e1waeSRYYKixacr0TfHz55Y=yvQ@mail.gmail.com>
	<CAJGK=o5H4Dwg-TchpvPe0SB=Y6-wwSFZoAF4i-i=WJAUV3gpXg@mail.gmail.com>
	<CA+LmXJAMfAm9v5=Vm4Gw9AOr_8rFRL5L7T65o9Q6LhiP3kh48g@mail.gmail.com>
	<CAJGK=o5qXp34aHSY79xyDj_0LQD3wzYyX0qiF2JZfNa90H36Eg@mail.gmail.com>
	<CA+LmXJDXgTJh+ReKgxPazbZt5hhxnO+Tf56fYrzUHe3FfTX_+g@mail.gmail.com>
	<CAJGK=o7XQxp0gdJbcZkyS_jwvYgU8wBcG7SfhjE9n=jWRRbdSw@mail.gmail.com>
	<CAGRA8uPEW5k5fs5PYOcHFzPBSjm_FV2f4FbT=BCSiCJo8iMNsA@mail.gmail.com>
	<CAJGK=o4dxJxwyo8HMZ71Zc+O5yA-JNdMC3hKBejzPC55HHz8RA@mail.gmail.com>
	<CAGRA8uPr=D-if+4OkYEtzuXc9CaWmAUC-jO9h3D2sXU+8pXsFQ@mail.gmail.com>
	<CAJGK=o6TsyteQ4i2tDO4dH8ahfQhdoa2baVxF3wxAEUk+UEOmg@mail.gmail.com>
	<CAJGK=o5EPcLTLhSKpOSOaox9DzVVZrP3N0Z-aiNJysWsha_KEw@mail.gmail.com>
	<CAGRA8uO1CNVe4AavYBRQaJRWsVgEM+B216YZNPK_OmsLXycTJw@mail.gmail.com>
	<CAJGK=o6mHmMnRi9RdkBrbxGmjsSsGG7BbBr6U7M-xTXCaiyL2A@mail.gmail.com>
Date: Sat, 31 Jan 2015 13:54:44 -0800
Message-ID: 
 <CAGRA8uObEyn1oL=k2qykRu8cjfdaUm+LNO5P=qDM5Gz90Wqwww@mail.gmail.com>
Subject: Re: AURORA-507
From: Bill Farner <wfarner@apache.org>
To: "dev@aurora.incubator.apache.org" <dev@aurora.incubator.apache.org>
Content-Type: multipart/alternative; boundary=047d7bdcabaae0bcad050df9c4dd

--047d7bdcabaae0bcad050df9c4dd
Content-Type: text/plain; charset=UTF-8

Locks are currently centered around job updates.  If you trace the rpc
name into the client code, you will wind up at a sub command cancel-update,
which does as you describe.

On Saturday, January 31, 2015, Arunabha Ghosh <arunabha.gh@gmail.com> wrote:

> Thanks, Bill. What happens if the client requesting a lock dies ? Is there
> a force release mechanism ?
>
> Thanks,
> Arunabha
>
> On Fri, Jan 30, 2015 at 5:09 PM, Bill Farner <wfarner@apache.org
> <javascript:;>> wrote:
>
> > Correct, they are global and durable.
> >
> > -=Bill
> >
> > On Fri, Jan 30, 2015 at 5:05 PM, Arunabha Ghosh <arunabha.gh@gmail.com
> <javascript:;>>
> > wrote:
> >
> > > Also, are these locks global ? is that why they are written to the WAL
> ?
> > >
> > > On Fri, Jan 30, 2015 at 5:04 PM, Arunabha Ghosh <arunabha.gh@gmail.com
> <javascript:;>>
> > > wrote:
> > >
> > > > Cool, that would be much simpler :-)
> > > >
> > > > On Fri, Jan 30, 2015 at 11:22 AM, Bill Farner <wfarner@apache.org
> <javascript:;>>
> > > wrote:
> > > >
> > > >> I think the guidance i gave you early on may have been overkill.
> You
> > > >> might
> > > >> consider isolating your change to SchedulerThriftInterface, and
> filter
> > > the
> > > >> result of lockManager.getLocks() to determine if the entity is
> locked
> > > >> (based on the lock key).  This is obviously not optimal w.r.t.
> > > >> performance,
> > > >> but given that the number of locks is likely to be quite small, an
> > O(n)
> > > >> operation should be fine to save the complexity.
> > > >>
> > > >> -=Bill
> > > >>
> > > >> On Thu, Jan 29, 2015 at 10:25 PM, Arunabha Ghosh <
> > arunabha.gh@gmail.com <javascript:;>
> > > >
> > > >> wrote:
> > > >>
> > > >> > Hmm, I'm happy to look at AURORA-189 first, but I'd still like to
> > keep
> > > >> > working on 507 if possible. From reading up on MyBatis, it does
> not
> > > >> look to
> > > >> > be very complicated.
> > > >> >
> > > >> > Does all persistent lock state end up in the DbLockStore
> eventually
> > ?
> > > >> i.e
> > > >> > does WriteAheadStorage call DbLockStore internally (through the
> > > >> > LockStore.Mutable member) ?
> > > >> >
> > > >> > Also, is the approach I outlined feasible ?
> > > >> >
> > > >> > Thanks,
> > > >> > Arunabha
> > > >> >
> > > >> > On Thu, Jan 29, 2015 at 11:16 AM, Bill Farner <wfarner@apache.org
> <javascript:;>>
> > > >> wrote:
> > > >> >
> > > >> > > The missing detail is that MyBatis dynamically creates an
> > > >> implementation
> > > >> > of
> > > >> > > LockMapper based on LockMapper.xml [1].
> > > >> > >
> > > >> > > However, i'm now realizing that this should probably not be
> > > classified
> > > >> > as a
> > > >> > > newbie ticket, since the the plumbing goes pretty deep, and
> > > knowledge
> > > >> of
> > > >> > > several technologies is necessary.  If you're just looking for
> > > >> something
> > > >> > to
> > > >> > > cut you teeth, i suggest you look past this one.  I'll suggest
> > this
> > > >> as an
> > > >> > > alternative: https://issues.apache.org/jira/browse/AURORA-189
> > > >> > >
> > > >> > >
> > > >> > > [1]
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://github.com/apache/incubator-aurora/blob/master/src/main/resources/org/apache/aurora/scheduler/storage/db/LockMapper.xml
> > > >> > >
> > > >> > > -=Bill
> > > >> > >
> > > >> > > On Wed, Jan 28, 2015 at 10:22 PM, Arunabha Ghosh <
> > > >> arunabha.gh@gmail.com <javascript:;>>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Thanks Chris !
> > > >> > > >
> > > >> > > > After a look at the code here's the approach I'm considering
> > > >> > > >
> > > >> > > > 1) Override releaseLock in LockManager to take an extra
> > parameter
> > > >> which
> > > >> > > can
> > > >> > > > be used to return extra information.
> > > >> > > > 2) Add a doesLockExist method to the LockStore interface and
> > > >> implement
> > > >> > > that
> > > >> > > > in DbLockStore and WriteAheadStorage
> > > >> > > > 3) Use the doesLockExist method in the implementations of the
> > > >> > overridden
> > > >> > > > releaseLock method in LockManager
> > > >> > > > 2) Make SchedulerThriftInterface call the overloaded version
> of
> > > >> > > releaseLock
> > > >> > > > in LockManager and add extra information to the response if
> > > needed.
> > > >> > > >
> > > >> > > > Does that seem like a reasonable way to proceed ?
> > > >> > > >
> > > >> > > > I have a few questions
> > > >> > > >
> > > >> > > > 1) I'm also a little bit mystified by the LockMapper interface
> > > which
> > > >> > does
> > > >> > > > not seem to be implemented anywhere. DbLockStore uses this
> > > >> interface.
> > > >> > > > 2) WriteAheadStorage uses a LockStore.Mutable member to
> > implement
> > > >> the
> > > >> > > > actual removeLock method, however I'm unable to figure out
> which
> > > >> > > > implementation of LockStore.Mutable is being used as the
> > > >> > > WriteAheadStorage
> > > >> > > > class is instantiated in LogStorage which is instantiated by
> > Guice
> > > >> > > (Blast!,
> > > >> > > > I thought I'd seen the last of Guice)
> > > >> > > >
> > > >> > > > Thanks,
> > > >> > > > Arunabha
> > > >> > > >
> > > >> > > >
> > > >> > > > On Tue, Jan 27, 2015 at 9:07 PM, Chris Lambert
> > > >> > > > <clambert@twitter.com.invalid
> > > >> > > > > wrote:
> > > >> > > >
> > > >> > > > > Updated.  Enjoy!
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Tue, Jan 27, 2015 at 8:39 PM, Arunabha Ghosh <
> > > >> > arunabha.gh@gmail.com <javascript:;>
> > > >> > > >
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Ok, my JIRA username is 'arunabha'
> > > >> > > > > >
> > > >> > > > > > On Tue, Jan 27, 2015 at 7:29 PM, Chris Lambert
> > > >> > > > > > <clambert@twitter.com.invalid
> > > >> > > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > Bill, not sure how I should get a JIRA username. I
> > signed
> > > up
> > > >> > for
> > > >> > > > > > > > Reviewboard though.
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > I think you can just use the signup link on the login
> page
> > > at
> > > >> > > > > > > issues.apache.org/jira
> > > >> > > > > > > <
> > https://issues.apache.org/jira/secure/Signup!default.jspa
> > > >.
> > > >> > > > > > >
> > > >> > > > > > > Chris
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > > On Tue, Jan 27, 2015 at 5:01 PM, Bill Farner <
> > > >> > wfarner@apache.org <javascript:;>
> > > >> > > >
> > > >> > > > > > wrote:
> > > >> > > > > > > >
> > > >> > > > > > > > > I don't believe we have really discussed the future
> of
> > > >> these
> > > >> > > > RPCs,
> > > >> > > > > > and
> > > >> > > > > > > > > specifically whether we will remove the ability for
> > > users
> > > >> to
> > > >> > > > > > implement
> > > >> > > > > > > > > client-side updaters.  I think a broader discussion
> on
> > > the
> > > >> > > future
> > > >> > > > > of
> > > >> > > > > > > job
> > > >> > > > > > > > > updates is warranted if you'd like to propose
> removing
> > > >> that
> > > >> > set
> > > >> > > > of
> > > >> > > > > > > RPCs.
> > > >> > > > > > > > >
> > > >> > > > > > > > > -=Bill
> > > >> > > > > > > > >
> > > >> > > > > > > > > On Tue, Jan 27, 2015 at 4:59 PM, Maxim Khutornenko <
> > > >> > > > > maxim@apache.org <javascript:;>
> > > >> > > > > > >
> > > >> > > > > > > > > wrote:
> > > >> > > > > > > > >
> > > >> > > > > > > > > > Thanks for reaching out! We are most likely going
> to
> > > >> > > deprecate
> > > >> > > > > the
> > > >> > > > > > > > > > acquireLock and releaseLock RPCs once the client
> > > >> updater is
> > > >> > > > > removed
> > > >> > > > > > > > > > (AURORA-785).
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > There are plenty of other entry level items to
> chose
> > > >> from
> > > >> > [1]
> > > >> > > > > > though
> > > >> > > > > > > > > > and we would greatly appreciate your help!
> > > >> > > > > > > > > >
> > > >> > > > > > > > > >
> > > >> > > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://issues.apache.org/jira/browse/AURORA-1064?jql=project%20%3D%20AURORA%20AND%20status%20in%20(Open%2C%20Accepted)%20AND%20labels%20in%20(newbie)
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > Thanks,
> > > >> > > > > > > > > > Maxim
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > On Tue, Jan 27, 2015 at 4:49 PM, Arunabha Ghosh <
> > > >> > > > > > > arunabha.gh@gmail.com <javascript:;>
> > > >> > > > > > > > >
> > > >> > > > > > > > > > wrote:
> > > >> > > > > > > > > > > Is anyone working on AURORA-507
> > > >> > > > > > > > > > > <
> https://issues.apache.org/jira/browse/AURORA-507
> > >
> > > ?
> > > >> If
> > > >> > > not
> > > >> > > > > I'd
> > > >> > > > > > > like
> > > >> > > > > > > > > to
> > > >> > > > > > > > > > > start working on it. What would be a good place
> to
> > > >> start
> > > >> > ?
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > Thanks,
> > > >> > > > > > > > > > > Arunabha
> > > >> > > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>


-- 
-=Bill

--047d7bdcabaae0bcad050df9c4dd--