aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Farner <wfar...@apache.org>
Subject Re: [PROPOSAL] Revisit task ID format
Date Fri, 19 Feb 2016 23:14:02 GMT
It has now been submitted.

On Fri, Feb 19, 2016 at 9:32 AM, Chris Lambert <chrislambert@gmail.com>
wrote:

> 'Morning,
>
> What's the eta for this patch, Bill?  Planning to commit soon?
>
> Chris
>
>
> On Wed, Jan 27, 2016 at 8:53 PM, Bill Farner <wfarner@apache.org> wrote:
>
> > Thanks for the input, folks.  Patch is up here:
> > https://reviews.apache.org/r/42896/
> >
> > On Wed, Jan 27, 2016 at 8:09 PM, Maxim Khutornenko <maxim@apache.org>
> > wrote:
> >
> > > You're right, ignore me. I guess I was mourning the loss of
> > StorageBackfill
> > > too hard :) Obviously, we don't have to force this change over existing
> > > tasks and let them die out naturally. Some user scraping tools may be
> > > broken due to this but we always advised against taking a dependency on
> > our
> > > task ID format.
> > >
> > > +1 to the proposal.
> > >
> > > On Wed, Jan 27, 2016 at 5:46 PM, Bill Farner <wfarner@apache.org>
> wrote:
> > >
> > > > I don't believe backwards compatibility is an issue here.  This would
> > be
> > > an
> > > > alteration to generation of new task IDs.  AFAIK we don't do anything
> > > that
> > > > requires comprehension or synthesis of previously-generated task IDs.
> > > >
> > > > On Wed, Jan 27, 2016 at 5:39 PM, Maxim Khutornenko <maxim@apache.org
> >
> > > > wrote:
> > > >
> > > > > What's the cluster upgrade story going to look like? Since task IDs
> > are
> > > > > used as unique identifiers for Mesos, I expect this change would
> > > require
> > > > > rebooting the entire cluster to the new format? I am not really
> sure
> > > how
> > > > > this can be done in a graceful manner without sending the entire
> > > cluster
> > > > to
> > > > > LOST on first reconciliation run.
> > > > >
> > > > > Unless we have answers to the above, I am -1 to this proposal. The
> > > > benefits
> > > > > don't seem significant enough to offset the pain of migrating to
> the
> > > new
> > > > > format.
> > > > >
> > > > > On Tue, Jan 26, 2016 at 6:25 PM, Mauricio Garavaglia <
> > > > > mauriciogaravaglia@gmail.com> wrote:
> > > > >
> > > > > > +1 to dropping the timestamp.
> > > > > >
> > > > > > Agree that having the jobkey at hand has been helpful for
> > debugging.
> > > > > >
> > > > > > On Tue, Jan 26, 2016 at 10:39 PM, Zameer Manji <
> zmanji@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > +1 to removing the timestamp.
> > > > > > >
> > > > > > > The timestamp has not provided me with any benefit as an
> > operator.
> > > > The
> > > > > > > mangled jobkey and UUID have been very useful in grepping
logs
> > and
> > > > > > > diagnosing failing jobs.
> > > > > > >
> > > > > > > On Tue, Jan 26, 2016 at 3:49 PM, Zhitao Li <
> > zhitaoli.cs@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > +1 for dropping the time and keeping the mangled jobkey.
> Unless
> > > we
> > > > > are
> > > > > > > sure
> > > > > > > > that all internal logging of Mesos master and agent
contains
> an
> > > > > > > identifier
> > > > > > > > with user some user generated data, changing it to
UUID will
> > make
> > > > > adhoc
> > > > > > > > debugging through Mesos logging harder.
> > > > > > > >
> > > > > > > > On Tue, Jan 26, 2016 at 3:17 PM, Erb, Stephan <
> > > > > > > Stephan.Erb@blue-yonder.com
> > > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1 for dropping the timestamp
> > > > > > > > >
> > > > > > > > > However, I am not sure regarding the mangled
jobkey. It
> tends
> > > to
> > > > > make
> > > > > > > it
> > > > > > > > > easier to correlate Mesos tasks to Aurora jobs
when
> skimming
> > > log
> > > > > > files,
> > > > > > > > > viewing the Mesos-UI or even when using the Thermos
[1]. I
> > > guess
> > > > > the
> > > > > > > > > traceability of all of those usecases could be
improved,
> but
> > > that
> > > > > > would
> > > > > > > > > probably additional work.
> > > > > > > > >
> > > > > > > > > [1]
> > > > > > > > >
> > > > > > >
> > > > >
> > >
> https://github.com/apache/aurora/blob/master/docs/images/runningtask.png
> > > > > > > > > ________________________________________
> > > > > > > > > From: Bill Farner <wfarner@apache.org>
> > > > > > > > > Sent: Wednesday, January 27, 2016 12:03 AM
> > > > > > > > > To: dev@aurora.apache.org
> > > > > > > > > Subject: [PROPOSAL] Revisit task ID format
> > > > > > > > >
> > > > > > > > > Context: a task ID is a unique identifier for
a task.
> Aurora
> > > and
> > > > > > Mesos
> > > > > > > > > both require this uniqueness.  Within mesos,
frameworks are
> > > > > required
> > > > > > to
> > > > > > > > > craft their own task IDs as they see fit.
> > > > > > > > >
> > > > > > > > > Our task ID format is currently [1]
> > > > > > > > >
> > > > > > > > > TIMESTAMP-ROLE-ENV-JOBNAME-INSTANCE-UUID
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > for an example:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> 1453847837931-vagrant-test-http_example_docker-1-a23f55e2-151f-409e-9cea-76ec79482533
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > In addition to guaranteed uniqueness, this format
has the
> > > benefit
> > > > > of
> > > > > > > > being
> > > > > > > > > somewhat human-friendly.  Within logs, it is
somewhat
> > possible
> > > to
> > > > > > > > partially
> > > > > > > > > recognize a task based solely on the text ID.
> > > > > > > > >
> > > > > > > > > *I would like to propose that we remove the TIMESTAMP-
> prefix
> > > > from
> > > > > > the
> > > > > > > > task
> > > > > > > > > ID.*  It was originally included so that task
IDs would be
> > > > > temporally
> > > > > > > > > sortable for scheduling prioritization.  At present,
tasks
> > are
> > > > not
> > > > > > > sorted
> > > > > > > > > using the ID.
> > > > > > > > >
> > > > > > > > > While proposing the above, i think it's also
prudent to
> take
> > > the
> > > > > > > > > opportunity to consider a complete overhaul of
the ID
> > contents.
> > > > > *An
> > > > > > > > > alternative approach would be to only use the
UUID.*  This
> > has
> > > > the
> > > > > > > > benefit
> > > > > > > > > of decoupling arbitrary user input from the various
ways
> task
> > > IDs
> > > > > are
> > > > > > > > used
> > > > > > > > > (as an example - mesos uses them in file names,
limiting
> > length
> > > > and
> > > > > > > > allowed
> > > > > > > > > characters).  Task IDs also become fixed width,
which
> offers
> > a
> > > > > (very)
> > > > > > > > > marginal memory reduction over the status quo,
and makes
> > > console
> > > > > line
> > > > > > > > > wrapping more consistent when perusing logs.
 Additionally,
> > it
> > > > > > eschews
> > > > > > > > the
> > > > > > > > > potential problem of users parsing task IDs and
coupling to
> > its
> > > > > > format.
> > > > > > > > >
> > > > > > > > > Any thoughts on this?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > [1]
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/TaskIdGenerator.java
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Cheers,
> > > > > > > >
> > > > > > > > Zhitao Li
> > > > > > > >
> > > > > > > > --
> > > > > > > > Zameer Manji
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message