cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Weisberg <ariel.weisb...@datastax.com>
Subject Re: Staging Branches
Date Thu, 07 May 2015 15:51:53 GMT
Hi,

I meant in the hypothetical case that we did this. There is going to be an
interim period where we wouldn't have this. The automation comes at the
expense of something else.

Ariel

On Thu, May 7, 2015 at 11:40 AM, Josh McKenzie <josh.mckenzie@datastax.com>
wrote:

> >
> > So who and when is going to implement the automation?
>
>
> I don't believe we have sufficient consensus that this is necessary to
> start doling out action-items for implementation.
>
> On Thu, May 7, 2015 at 10:16 AM, Ariel Weisberg <
> ariel.weisberg@datastax.com
> > wrote:
>
> > Hi,
> >
> > If it were automated I would have no problem with it. That would be less
> > work for me because the problems detected would occur anyways and have to
> > be dealt with by me. I just don't want to deal with extra steps and
> latency
> > manually.
> >
> > So who and when is going to implement the automation?
> >
> > Ariel
> >
> > On Thu, May 7, 2015 at 11:11 AM, Benedict Elliott Smith <
> > belliottsmith@datastax.com> wrote:
> >
> > > It's odd, because I honestly think this release process will be easier,
> > > since the stricter we make it the smoother it can become. It requires
> > well
> > > formed commits from everyone, and lets the committers asynchronously
> > > confirm their work, and for it to never be in question *who* needs to
> fix
> > > something, nor what the effect of their fixing it will be. It means we
> > can,
> > > as Ariel said, perform a bisect and honestly know its result is
> accurate.
> > > Small commits don't need to worry about fast-forwarding; in fact,
> nobody
> > > does. It can either be automated, or we can fast forward at a time that
> > > suits us. In which case the process is *the same* as it is currently.
> > >
> > > I have no interest in making the commit process harder.
> > >
> > >
> > > On Thu, May 7, 2015 at 3:59 PM, Jake Luciani <jakers@gmail.com> wrote:
> > >
> > > > Ok let's focus then on the idea that trunk is releasable.  Releasable
> > > > to me doesn't mean it can't contain a bad merge.
> > > >
> > > > It means it doesn't contain some untested and unstable feature.  We
> > > > can always "release from trunk" and we still have a release process.
> > > >
> > > > The idea that trunk must contain. a first time it hits the branch,
> > > > releasable code is way overboard
> > > >
> > > > On Thu, May 7, 2015 at 10:50 AM, Benedict Elliott Smith
> > > > <belliottsmith@datastax.com> wrote:
> > > > >>
> > > > >> This breaks your model of applying every commit ref by ref.
> > > > >
> > > > >
> > > > > How? The rebase only affects commits after the "real" branch, so
it
> > > still
> > > > > cleanly fast forwards?
> > > > >
> > > > > Merging is *hard*. Especially 2.1 -> 3.0, with many breaking API
> > > changes
> > > > > (this is before 8099, which is going to make a *world* of hurt, and
> > > will
> > > > > stick around for a year). It is *very* easy to break things, with
> > even
> > > > the
> > > > > utmost care.
> > > > >
> > > > > On Thu, May 7, 2015 at 3:46 PM, Jake Luciani <jakers@gmail.com>
> > wrote:
> > > > >
> > > > >> You then fetch and repair
> > > > >> your local version and try again.
> > > > >>
> > > > >> This breaks your model of applying every commit ref by ref.
> > > > >>
> > > > >> I'm all for trying to avoid extra work/stability but we already
> have
> > > > >> added a layer of testing every change before commit.  I'm not
> going
> > to
> > > > >> accept we need to also add a layer of testing before every merge.
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Thu, May 7, 2015 at 10:36 AM, Benedict Elliott Smith
> > > > >> <belliottsmith@datastax.com> wrote:
> > > > >> >>
> > > > >> >> wouldn't you need to force push?
> > > > >> >
> > > > >> >
> > > > >> > git push --force-with-lease
> > > > >> >
> > > > >> > This works essentially like CAS; if the remote repositories
are
> > not
> > > > the
> > > > >> > same as the one you have modified, it will fail. You then
fetch
> > and
> > > > >> repair
> > > > >> > your local version and try again.
> > > > >> >
> > > > >> > So what does this buy us?
> > > > >> >
> > > > >> >
> > > > >> > This buys us a clean development process. We bought into
"always
> > > > >> > releasable". It's already a tall order; if we start weakening
> the
> > > > >> > constraints before we even get started, I am unconvinced
we will
> > > > >> > successfully deliver. A monthly release cycle requires *strict*
> > > > >> processes,
> > > > >> > not *almost* strict, or strict*ish*.
> > > > >> >
> > > > >> > Something that could also help make a more streamlined process:
> if
> > > > actual
> > > > >> > commits were constructed on development branches ready for
> commit,
> > > > with a
> > > > >> > proper commit message and CHANGES.txt updated. Even more
> ideally:
> > > with
> > > > >> git
> > > > >> > rerere data for merging up to each of the branches. If we
had
> > that,
> > > > and
> > > > >> > each of the branches had been tested in CI, we would be
much
> > closer
> > > > than
> > > > >> we
> > > > >> > are currently, as the risk-at-commit is minimized.
> > > > >> >
> > > > >> > On Thu, May 7, 2015 at 2:48 PM, Jake Luciani <jakers@gmail.com>
> > > > wrote:
> > > > >> >
> > > > >> >> git rebase -i trunk_staging <ref~1>
> > > > >> >> fix the problem
> > > > >> >> git rebase --continue
> > > > >> >>
> > > > >> >> In this situation, if there was an untested follow on
commit
> > > wouldn't
> > > > >> >> you need to force push?
> > > > >> >>
> > > > >> >> On Thu, May 7, 2015 at 9:28 AM, Benedict Elliott Smith
> > > > >> >> <belliottsmith@datastax.com> wrote:
> > > > >> >> >>
> > > > >> >> >> If we do it, we'll end up in weird situations
which will be
> > > > annoying
> > > > >> for
> > > > >> >> >> everyone
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > Such as? I'm not disputing, but if we're to assess
the
> relative
> > > > >> >> > strengths/weaknesses, we need to have specifics
to discuss.
> > > > >> >> >
> > > > >> >> > If we do go with this suggestion, we will most
likely want to
> > > > enable a
> > > > >> >> > shared git rerere cache, so that rebasing is not
painful when
> > > there
> > > > >> are
> > > > >> >> > future commits.
> > > > >> >> >
> > > > >> >> > If instead we go with "repairing" commits, we cannot
have a
> > > > "queue" of
> > > > >> >> > things to merge up to. Say you have a string of
commits
> waiting
> > > for
> > > > >> >> > approval C1 to C4; you made C1, and it broke something.
You
> > > > introduce
> > > > >> C5
> > > > >> >> to
> > > > >> >> > fix it, but the tests are still broken. Did you
not really
> fix
> > > it?
> > > > Or
> > > > >> >> > perhaps one of C2 to C4 are to blame, but which?
And have you
> > > > >> >> accidentally
> > > > >> >> > broken *them* with your commit? Who knows. Either
way, we
> > > > definitely
> > > > >> >> cannot
> > > > >> >> > fast forward. At the very best we can hope that
the new merge
> > did
> > > > not
> > > > >> >> > conflict or mess up the other people's C2 to C4
commits, and
> > they
> > > > >> have to
> > > > >> >> > now merge on top. But what if another merge comes
in, C6, in
> > the
> > > > >> >> meantime;
> > > > >> >> > and C2 really did also break the tests in some
way; how do we
> > > > >> determine
> > > > >> >> C2
> > > > >> >> > was to blame, and not C6, or C3 or C4? What do
the committers
> > for
> > > > >> each of
> > > > >> >> > these do? We end up in a lengthy tussle, and aren't
able to
> > > commit
> > > > >> any of
> > > > >> >> > these to the mainline until all of them are resolved.
Really
> we
> > > > have
> > > > >> to
> > > > >> >> > prevent any merges to the staging repository until
the
> mistakes
> > > are
> > > > >> >> fixed.
> > > > >> >> > Since our races in these scenario are the length
of time
> taken
> > > for
> > > > >> cassci
> > > > >> >> > to vet them, these problems are much more likely
than current
> > > race
> > > > to
> > > > >> >> > commit.
> > > > >> >> >
> > > > >> >> > In the scheme I propose, in this scenario, the
person who
> broke
> > > the
> > > > >> build
> > > > >> >> > rebases everyone's branches to his now fixed commit,
and the
> > next
> > > > >> broken
> > > > >> >> > commit gets blamed, and all other commits being
merged in on
> > top
> > > > can
> > > > >> go
> > > > >> >> in
> > > > >> >> > smoothly. The only pain point I can think of is
the
> > multi-branch
> > > > >> rebase,
> > > > >> >> > but this is solved by git rerere.
> > > > >> >> >
> > > > >> >> > I agree running tests is painful, but at least
for the build,
> > > this
> > > > >> should
> > > > >> >> >> be the responsibility of the committer to build
before
> merging
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > Why make the distinction if we're going to have
staging
> > commits?
> > > > It's
> > > > >> a
> > > > >> >> bit
> > > > >> >> > of a waste of time to run three ant real-clean
&& ant tasks,
> > and
> > > > >> >> increases
> > > > >> >> > the race window for merging (which is painful whether
or not
> > > > involves
> > > > >> a
> > > > >> >> > rebase), and it is not a *typical* occurrence ("alarming"
is
> > > > >> subjective)
> > > > >> >> >
> > > > >> >> > On Thu, May 7, 2015 at 2:12 PM, Sylvain Lebresne
<
> > > > >> sylvain@datastax.com>
> > > > >> >> > wrote:
> > > > >> >> >
> > > > >> >> >> > If one of them breaks, we go and edit
the _staging branch
> in
> > > > place
> > > > >> to
> > > > >> >> >> correct
> > > > >> >> >> > the problem, and let CI run again.
> > > > >> >> >>
> > > > >> >> >> I would strongly advise against *in place*
edits. If we do
> it,
> > > > we'll
> > > > >> >> end up
> > > > >> >> >> in
> > > > >> >> >> weird situations which will be annoying for
everyone.
> Editing
> > > > commits
> > > > >> >> that
> > > > >> >> >> have
> > > > >> >> >> been shared is almost always a bad idea and
that's
> especially
> > > true
> > > > >> for
> > > > >> >> >> branch
> > > > >> >> >> that will have some amount of concurrency like
those staging
> > > > >> branches.
> > > > >> >> >>
> > > > >> >> >> Even if such problems are rare, better to avoid
them in the
> > > first
> > > > >> place
> > > > >> >> by
> > > > >> >> >> simply
> > > > >> >> >> commit new "fixup" commits as we currently
do. Granted this
> > give
> > > > you
> > > > >> a
> > > > >> >> >> slightly
> > > > >> >> >> less clean history but to the best of my knowledge,
this
> > hasn't
> > > > been
> > > > >> a
> > > > >> >> pain
> > > > >> >> >> point so far.
> > > > >> >> >>
> > > > >> >> >> > wait for CI; all clear
> > > > >> >> >> >
> > > > >> >> >> > git checkout cassandra-2.0; git merge
> cassandra-2.0_staging
> > > > >> >> >> > git checkout cassandra-2.1; git merge
> cassandra-2.1_staging
> > > > >> >> >> > git checkout trunk; git merge trunk_staging
> > > > >> >> >> >
> > > > >> >> >> > This does introduce some extra steps to
the merge process
> > > > >> >> >>
> > > > >> >> >> If we do this, we should really automate that
last part
> (have
> > > the
> > > > CI
> > > > >> >> >> environment merge the staging branch to the
non-staging ones
> > on
> > > > >> >> success).
> > > > >> >> >>
> > > > >> >> >> > It seems if we want an "always releasable"
set of
> branches,
> > we
> > > > need
> > > > >> >> >> something
> > > > >> >> >> > along these lines.
> > > > >> >> >>
> > > > >> >> >> Agreed as far as having staging branches vetoed
by CI goes.
> > Less
> > > > sure
> > > > >> >> about
> > > > >> >> >> the edit-commit-in-place part as said above.
> > > > >> >> >>
> > > > >> >> >> > I certainly break tests by mistake, or
the build itself,
> > with
> > > > >> alarming
> > > > >> >> >> regularity.
> > > > >> >> >>
> > > > >> >> >> I agree running tests is painful, but at least
for the
> build,
> > > this
> > > > >> >> should
> > > > >> >> >> be
> > > > >> >> >> the responsibility of the committer to build
before merging.
> > We
> > > > all
> > > > >> >> forget
> > > > >> >> >> it from
> > > > >> >> >> times to times and that's ok, but it's not
ok if it's
> > > "alarmingly
> > > > >> >> regular".
> > > > >> >> >>
> > > > >> >> >> --
> > > > >> >> >> Sylvain
> > > > >> >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >> --
> > > > >> >> http://twitter.com/tjake
> > > > >> >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> http://twitter.com/tjake
> > > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > http://twitter.com/tjake
> > > >
> > >
> >
>
>
>
> --
> Joshua McKenzie
> DataStax -- The Apache Cassandra Company
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message