aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Farner <wfar...@apache.org>
Subject Re: AURORA-507
Date Sat, 31 Jan 2015 21:54:44 GMT
Locks are currently centered around job updates.  If you trace the rpc
name into the client code, you will wind up at a sub command cancel-update,
which does as you describe.

On Saturday, January 31, 2015, Arunabha Ghosh <arunabha.gh@gmail.com> wrote:

> Thanks, Bill. What happens if the client requesting a lock dies ? Is there
> a force release mechanism ?
>
> Thanks,
> Arunabha
>
> On Fri, Jan 30, 2015 at 5:09 PM, Bill Farner <wfarner@apache.org
> <javascript:;>> wrote:
>
> > Correct, they are global and durable.
> >
> > -=Bill
> >
> > On Fri, Jan 30, 2015 at 5:05 PM, Arunabha Ghosh <arunabha.gh@gmail.com
> <javascript:;>>
> > wrote:
> >
> > > Also, are these locks global ? is that why they are written to the WAL
> ?
> > >
> > > On Fri, Jan 30, 2015 at 5:04 PM, Arunabha Ghosh <arunabha.gh@gmail.com
> <javascript:;>>
> > > wrote:
> > >
> > > > Cool, that would be much simpler :-)
> > > >
> > > > On Fri, Jan 30, 2015 at 11:22 AM, Bill Farner <wfarner@apache.org
> <javascript:;>>
> > > wrote:
> > > >
> > > >> I think the guidance i gave you early on may have been overkill.
> You
> > > >> might
> > > >> consider isolating your change to SchedulerThriftInterface, and
> filter
> > > the
> > > >> result of lockManager.getLocks() to determine if the entity is
> locked
> > > >> (based on the lock key).  This is obviously not optimal w.r.t.
> > > >> performance,
> > > >> but given that the number of locks is likely to be quite small, an
> > O(n)
> > > >> operation should be fine to save the complexity.
> > > >>
> > > >> -=Bill
> > > >>
> > > >> On Thu, Jan 29, 2015 at 10:25 PM, Arunabha Ghosh <
> > arunabha.gh@gmail.com <javascript:;>
> > > >
> > > >> wrote:
> > > >>
> > > >> > Hmm, I'm happy to look at AURORA-189 first, but I'd still like
to
> > keep
> > > >> > working on 507 if possible. From reading up on MyBatis, it does
> not
> > > >> look to
> > > >> > be very complicated.
> > > >> >
> > > >> > Does all persistent lock state end up in the DbLockStore
> eventually
> > ?
> > > >> i.e
> > > >> > does WriteAheadStorage call DbLockStore internally (through the
> > > >> > LockStore.Mutable member) ?
> > > >> >
> > > >> > Also, is the approach I outlined feasible ?
> > > >> >
> > > >> > Thanks,
> > > >> > Arunabha
> > > >> >
> > > >> > On Thu, Jan 29, 2015 at 11:16 AM, Bill Farner <wfarner@apache.org
> <javascript:;>>
> > > >> wrote:
> > > >> >
> > > >> > > The missing detail is that MyBatis dynamically creates an
> > > >> implementation
> > > >> > of
> > > >> > > LockMapper based on LockMapper.xml [1].
> > > >> > >
> > > >> > > However, i'm now realizing that this should probably not
be
> > > classified
> > > >> > as a
> > > >> > > newbie ticket, since the the plumbing goes pretty deep,
and
> > > knowledge
> > > >> of
> > > >> > > several technologies is necessary.  If you're just looking
for
> > > >> something
> > > >> > to
> > > >> > > cut you teeth, i suggest you look past this one.  I'll suggest
> > this
> > > >> as an
> > > >> > > alternative: https://issues.apache.org/jira/browse/AURORA-189
> > > >> > >
> > > >> > >
> > > >> > > [1]
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://github.com/apache/incubator-aurora/blob/master/src/main/resources/org/apache/aurora/scheduler/storage/db/LockMapper.xml
> > > >> > >
> > > >> > > -=Bill
> > > >> > >
> > > >> > > On Wed, Jan 28, 2015 at 10:22 PM, Arunabha Ghosh <
> > > >> arunabha.gh@gmail.com <javascript:;>>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Thanks Chris !
> > > >> > > >
> > > >> > > > After a look at the code here's the approach I'm considering
> > > >> > > >
> > > >> > > > 1) Override releaseLock in LockManager to take an extra
> > parameter
> > > >> which
> > > >> > > can
> > > >> > > > be used to return extra information.
> > > >> > > > 2) Add a doesLockExist method to the LockStore interface
and
> > > >> implement
> > > >> > > that
> > > >> > > > in DbLockStore and WriteAheadStorage
> > > >> > > > 3) Use the doesLockExist method in the implementations
of the
> > > >> > overridden
> > > >> > > > releaseLock method in LockManager
> > > >> > > > 2) Make SchedulerThriftInterface call the overloaded
version
> of
> > > >> > > releaseLock
> > > >> > > > in LockManager and add extra information to the response
if
> > > needed.
> > > >> > > >
> > > >> > > > Does that seem like a reasonable way to proceed ?
> > > >> > > >
> > > >> > > > I have a few questions
> > > >> > > >
> > > >> > > > 1) I'm also a little bit mystified by the LockMapper
interface
> > > which
> > > >> > does
> > > >> > > > not seem to be implemented anywhere. DbLockStore uses
this
> > > >> interface.
> > > >> > > > 2) WriteAheadStorage uses a LockStore.Mutable member
to
> > implement
> > > >> the
> > > >> > > > actual removeLock method, however I'm unable to figure
out
> which
> > > >> > > > implementation of LockStore.Mutable is being used as
the
> > > >> > > WriteAheadStorage
> > > >> > > > class is instantiated in LogStorage which is instantiated
by
> > Guice
> > > >> > > (Blast!,
> > > >> > > > I thought I'd seen the last of Guice)
> > > >> > > >
> > > >> > > > Thanks,
> > > >> > > > Arunabha
> > > >> > > >
> > > >> > > >
> > > >> > > > On Tue, Jan 27, 2015 at 9:07 PM, Chris Lambert
> > > >> > > > <clambert@twitter.com.invalid
> > > >> > > > > wrote:
> > > >> > > >
> > > >> > > > > Updated.  Enjoy!
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Tue, Jan 27, 2015 at 8:39 PM, Arunabha Ghosh
<
> > > >> > arunabha.gh@gmail.com <javascript:;>
> > > >> > > >
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Ok, my JIRA username is 'arunabha'
> > > >> > > > > >
> > > >> > > > > > On Tue, Jan 27, 2015 at 7:29 PM, Chris Lambert
> > > >> > > > > > <clambert@twitter.com.invalid
> > > >> > > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > Bill, not sure how I should get
a JIRA username. I
> > signed
> > > up
> > > >> > for
> > > >> > > > > > > > Reviewboard though.
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > I think you can just use the signup
link on the login
> page
> > > at
> > > >> > > > > > > issues.apache.org/jira
> > > >> > > > > > > <
> > https://issues.apache.org/jira/secure/Signup!default.jspa
> > > >.
> > > >> > > > > > >
> > > >> > > > > > > Chris
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > > On Tue, Jan 27, 2015 at 5:01 PM,
Bill Farner <
> > > >> > wfarner@apache.org <javascript:;>
> > > >> > > >
> > > >> > > > > > wrote:
> > > >> > > > > > > >
> > > >> > > > > > > > > I don't believe we have really
discussed the future
> of
> > > >> these
> > > >> > > > RPCs,
> > > >> > > > > > and
> > > >> > > > > > > > > specifically whether we will
remove the ability for
> > > users
> > > >> to
> > > >> > > > > > implement
> > > >> > > > > > > > > client-side updaters.  I think
a broader discussion
> on
> > > the
> > > >> > > future
> > > >> > > > > of
> > > >> > > > > > > job
> > > >> > > > > > > > > updates is warranted if you'd
like to propose
> removing
> > > >> that
> > > >> > set
> > > >> > > > of
> > > >> > > > > > > RPCs.
> > > >> > > > > > > > >
> > > >> > > > > > > > > -=Bill
> > > >> > > > > > > > >
> > > >> > > > > > > > > On Tue, Jan 27, 2015 at 4:59
PM, Maxim Khutornenko <
> > > >> > > > > maxim@apache.org <javascript:;>
> > > >> > > > > > >
> > > >> > > > > > > > > wrote:
> > > >> > > > > > > > >
> > > >> > > > > > > > > > Thanks for reaching out!
We are most likely going
> to
> > > >> > > deprecate
> > > >> > > > > the
> > > >> > > > > > > > > > acquireLock and releaseLock
RPCs once the client
> > > >> updater is
> > > >> > > > > removed
> > > >> > > > > > > > > > (AURORA-785).
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > There are plenty of other
entry level items to
> chose
> > > >> from
> > > >> > [1]
> > > >> > > > > > though
> > > >> > > > > > > > > > and we would greatly
appreciate your help!
> > > >> > > > > > > > > >
> > > >> > > > > > > > > >
> > > >> > > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://issues.apache.org/jira/browse/AURORA-1064?jql=project%20%3D%20AURORA%20AND%20status%20in%20(Open%2C%20Accepted)%20AND%20labels%20in%20(newbie)
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > Thanks,
> > > >> > > > > > > > > > Maxim
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > On Tue, Jan 27, 2015
at 4:49 PM, Arunabha Ghosh <
> > > >> > > > > > > arunabha.gh@gmail.com <javascript:;>
> > > >> > > > > > > > >
> > > >> > > > > > > > > > wrote:
> > > >> > > > > > > > > > > Is anyone working
on AURORA-507
> > > >> > > > > > > > > > > <
> https://issues.apache.org/jira/browse/AURORA-507
> > >
> > > ?
> > > >> If
> > > >> > > not
> > > >> > > > > I'd
> > > >> > > > > > > like
> > > >> > > > > > > > > to
> > > >> > > > > > > > > > > start working on
it. What would be a good place
> to
> > > >> start
> > > >> > ?
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > Thanks,
> > > >> > > > > > > > > > > Arunabha
> > > >> > > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>


-- 
-=Bill

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message