impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Apple <jbap...@cloudera.com>
Subject Re: Impala 3.x (and 2.x)
Date Fri, 12 Jan 2018 23:47:47 GMT
Often, this list seems to filter out images. Could you post it and send a
link?

Thanks for taking this on, Phil!

On Fri, Jan 12, 2018 at 3:15 PM, Philip Zeyliger <philip@cloudera.com>
wrote:

> I think most patches go to Gerrit branch 'master', which happens to
> identify itself as 3.0. (Or 3.x?).
>
> Here's a picture:
>
> [image: Inline image 1]
>
>
> With this, every time "cherrypick_and_push_to_asf.py" is run, it would
> first offer to cherrypick changes between master and 2.x. Then, it would
> offer push those cherrypicks to gerrit/2.x. After that, it continues on as
> before and offers to push changes to ASF. I think this maintains the
> invariant that pushing to ASF is only done with a human trigger. (We could
> also have step 1 be done by a Jenkins robot, since it's between Gerrit and
> Gerrit.)
>
> I looked at the How to Release page, and the main difference would be
> that, for a 2.x release, the $COMMIT_HASH_YOU_CHOSE would come from the 2.x
> branch, as would any cherrypicks.
>
> Does this match what you're thinking?
>
> Thanks!
>
> -- Philip
>
> On Fri, Jan 12, 2018 at 11:07 AM, Jim Apple <jbapple@cloudera.com> wrote:
>
>> Which gerrit branch were you thinking most patches would go to?
>>
>> If they go to 3.0, then push_to_asf.py would have to be amended to
>> push to gerrit, bypassing code review. I think that's possible, but
>> I'm not 100%.
>>
>> There is also security to think about, since the push_to_asf.py users
>> can push a few commits at a time, including ones they didn't author or
>> review.
>>
>> We'll also want to clarify
>> https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release and
>> keep it consistent with the git & gerrit statuses quo.
>>
>> On Fri, Jan 12, 2018 at 10:52 AM, Philip Zeyliger <philip@cloudera.com>
>> wrote:
>> > Hi!
>> >
>> >> Should we start tagging all candidates with a common label, e.g.
>> > include-in-v3?
>> >
>> > I agree with Lars's suggestion for tagging JIRAs with include-in-v3.
>> I've
>> > done so, and the relevant query is
>> > https://issues.apache.org/jira/issues/?jql=labels%20%3D%20in
>> clude-in-v3%20and%20project%3Dimpala
>> > .
>> >
>> >> What sort of process were you thinking of for the automation?
>> >
>> > I think amending push_to_asf.py, as you suggest, is a great idea. I
>> think
>> > we have a string ("not for 2.x") which can be used in commit messages to
>> > discourage the cherrypick for the changes we want to be exclusive until
>> we
>> > want to change the defaults in the other direction. (I.e., right now the
>> > string is "not for 2.x", but at some point the string may be "should be
>> > cherrypicked to 2.x".)
>> >
>> > I do think that we want to create a gerrit branch to allow 2.x-only
>> changes
>> > to be reviewed in the straight-forward fashion.
>> >
>> > -- Philip
>> >
>> >
>> >
>> > On Thu, Jan 11, 2018 at 9:31 AM, Jim Apple <jbapple@cloudera.com>
>> wrote:
>> >
>> >> I'm on-board with all of this. (I also would be OK delaying 3.0, if
>> >> that were the consensus).
>> >>
>> >> There is one issue in here I think we should dive into:
>> >>
>> >> > Both master and 2.x would be active, and, at least for the beginning,
>> >> > changes would automatically be pulled into the 2.x line, unless
>> >> explicitly
>> >> > blacklisted.
>> >>
>> >> What sort of process were you thinking of for the automation?
>> >>
>> >> Some context, starting from what we all likely already know:
>> >>
>> >> The bulk of the code review and pre-merge testing results are recorded
>> >> in gerrit. Once the pre-merge testing passes, a patch is cherry-picked
>> >> to the git repo hosted with gerrit. To get the patch to the Impala git
>> >> repo hosted by the ASF, bin/push_to_asf.py is run by a human who
>> >> supplies his or her ASF credentials. That script copies the commit to
>> >> the ASF git repo.
>> >>
>> >> Often, 2-3 commits will pile up in gerrit before some committer runs
>> >> that script and pushes them to ASF.
>> >>
>> >> We could edit that script (bin/push_to_asf.py) to help with the cherry
>> >> picks, so that each time a commit is made, the committer must say
>> >> whether the commit goes in 2.x, 3.0, or both, but the commits are
>> >> often made by people who didn't author the patches, so they may not be
>> >> sure which branch to go in.
>> >>
>> >> Additionally, gerrit code review is intimately tied to the git repo.
>> >> Gerrit runs a git repo under-the-hood, and I believe that the branch
>> >> on gerrit's git that changes are cherry-picked to after pre-merge
>> >> testing is identical to the Impala git repo hosted by the ASF - down
>> >> to the hashes, even. If we think 2.x and 3.0 will diverge enough that
>> >> we'll want different code reviews for different branches, then we
>> >> might want two different branches on gerrit, too.
>> >>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message