asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Maxon <ima...@uci.edu>
Subject Re: The Great Merge
Date Fri, 08 Apr 2016 02:58:39 GMT
Say if you had an AsterixDB change in a topic branch that had both
AsterixDB and Hyracks components, it'd be like:

git checkout (topic branch)
git fetch origin
git merge origin/master
(resolve conflicts)
git commit
git remote add hyracks-local file://(absolute path to your hyracks repo)
git fetch hyracks-local
git merge hyracks-local/(name of your hyracks topic branch)
(resolve conflicts)


On Thu, Apr 7, 2016 at 7:31 PM, Taewoo Kim <wangsaeu@gmail.com> wrote:

> @Ian:
>
> Could you show the process (the explicit command, if possible) step by
> step? I know you did. But, I have a hard time to apply your merge. Thank
> you. Steven's patch should work, also. But it didn't work on my branch. It
> looks like I miss something.
>
>
>
> Best,
> Taewoo
>
> On Thu, Apr 7, 2016 at 9:12 AM, Ian Maxon <imaxon@uci.edu> wrote:
>
> > Also, please don't push any new changes to Hyracks. You will need to
> merge
> > these into your asterix change or a new change if it is a hyracks change
> > with no asterixdb content. I have disabled the jenkins job that verifies
> > hyracks patches on Gerrit so hopefully nothing gets through.
> >
> > On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <imaxon@uci.edu> wrote:
> >
> > > Alright, everything is pushed. Please post here or start another thread
> > if
> > > you start to experience issues.
> > >
> > > On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <imaxon@uci.edu> wrote:
> > >
> > >> Just as a reminder, I'll be merging this tommorow since it seems like
> > >> everything should go fairly smoothly.
> > >>
> > >> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sjaco002@ucr.edu>
> wrote:
> > >>
> > >>> http://landbeforetime.wikia.com/wiki/Great_Valley
> > >>>
> > >>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sjaco002@ucr.edu>
> > wrote:
> > >>>
> > >>> > No, we are living in the GREAT valley :)
> > >>> > Steven
> > >>> >
> > >>> > On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dtabass@gmail.com>
> > wrote:
> > >>> >
> > >>> >> Sounds like things are GOOD!  Excellent.  (So not to be feared
> like
> > >>> the
> > >>> >> event that the name of this one keeps reminding me of:
> > >>> >> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
> > >>> >>
> > >>> >>
> > >>> >> On 4/4/16 1:12 PM, Steven Jacobs wrote:
> > >>> >>
> > >>> >>> It seems that I might be the only one concerned here,
but it
> seems
> > >>> like
> > >>> >>> there should be others, so I am continuing this thread.
> > >>> >>>
> > >>> >>> I modified the perl REGEX from Chris' summer solution,
and it
> > works!
> > >>> >>>
> > >>> >>> Once Ian has merged master:
> > >>> >>>
> > >>> >>> 1. On your local branch, find the *parent* of the first
commit
> you
> > >>> want
> > >>> >>> to
> > >>> >>> migrate onto the new master, e.g.
> > >>> >>> de6e0da24c26037967eb9a937d2c77c6c43e8761
> > >>> >>>
> > >>> >>> 2. Run this magic command:
> > >>> >>>
> > >>> >>>     git format-patch --stdout
> > >>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
> > >>> >>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
> > >>> >>>
> > >>> >>> 3. Now fetch master, and create a new local branch from
it:
> > >>> >>>
> > >>> >>>     git switch master; git pull; git checkout -B newbranch
> > >>> >>>
> > >>> >>> 4. Apply your tweaked patch:
> > >>> >>>
> > >>> >>>     git am /tmp/my.patch
> > >>> >>>
> > >>> >>>
> > >>> >>> This recognized ALL of my file moves/renames and applied
them
> > >>> correctly.
> > >>> >>> It
> > >>> >>> leaves only two issues:
> > >>> >>> 1) Something similar will probably need to be done for
Hyracks
> > >>> changes
> > >>> >>> 2) My pom changes didn't apply. This isn't so bad since
there are
> > >>> only a
> > >>> >>> few pom files total.
> > >>> >>>
> > >>> >>>
> > >>> >>> I hope this helps,
> > >>> >>> Steven
> > >>> >>>
> > >>> >>>
> > >>> >>>
> > >>> >>>
> > >>> >>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sjaco002@ucr.edu
> >
> > >>> wrote:
> > >>> >>>
> > >>> >>> Here is Chris's original solution to give context. I think
> changing
> > >>> the
> > >>> >>>> REGEX might be enough to re-use the solution:
> > >>> >>>>
> > >>> >>>> 1. On your local branch, find the *parent* of the
first commit
> you
> > >>> want
> > >>> >>>> to
> > >>> >>>> migrate onto the new master. If you were fully up-to-date
before
> > the
> > >>> >>>> repackaging commits went in, this will be Till's
> > >>> >>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so
I'll use
> that
> > >>> here.
> > >>> >>>>
> > >>> >>>> 2. Run this magic command:
> > >>> >>>>
> > >>> >>>>     git format-patch --stdout
> > >>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
> > >>> >>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
> > >>> >>>>
> > >>> >>>> 3. Now fetch the new master, and create a new local
branch from
> > it:
> > >>> >>>>
> > >>> >>>>     git switch master; git pull; git checkout -B newbranch
> > >>> >>>>
> > >>> >>>> 4. Apply your tweaked patch:
> > >>> >>>>
> > >>> >>>>     git am /tmp/my.patch
> > >>> >>>>
> > >>> >>>>
> > >>> >>>> Steven
> > >>> >>>>
> > >>> >>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <
> sjaco002@ucr.edu>
> > >>> >>>> wrote:
> > >>> >>>>
> > >>> >>>> I've tried doing this now on my branch.
> > >>> >>>>> As I feared, all of the files that are renamed/moved
become
> > >>> conflicts
> > >>> >>>>> (just a few hundred conflicts in my case 😑).
> > >>> >>>>> I'm wondering if we could use a similar technique
for what we
> did
> > >>> >>>>> during
> > >>> >>>>> the summer (for the apache change) to get around
this.
> > >>> >>>>>
> > >>> >>>>> Steven
> > >>> >>>>>
> > >>> >>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann
<
> tillw@apache.org>
> > >>> >>>>> wrote:
> > >>> >>>>>
> > >>> >>>>> I’m not sure I completely understand what you
are saying. Is
> > this a
> > >>> >>>>>> temporary state that will get cleaned up later
or is this
> > >>> supposed to
> > >>> >>>>>> stay this way (having "-fullstack" in the
names)?
> > >>> >>>>>>
> > >>> >>>>>> Thanks,
> > >>> >>>>>> Till
> > >>> >>>>>>
> > >>> >>>>>>
> > >>> >>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
> > >>> >>>>>>
> > >>> >>>>>> I'm not sure if it was necessary to rename
it, but the
> original
> > >>> issue
> > >>> >>>>>> is
> > >>> >>>>>>
> > >>> >>>>>>> that the hyracks repo itself has a folder
named hyracks, that
> > >>> >>>>>>> contains
> > >>> >>>>>>> hyracks. I thought this might confuse
git if I did something
> > like
> > >>> >>>>>>> make a
> > >>> >>>>>>> new temporary folder, move everything
into that, and then
> > rename
> > >>> it
> > >>> >>>>>>> to
> > >>> >>>>>>> 'hyracks'.
> > >>> >>>>>>>
> > >>> >>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till
Westmann <
> > tillw@apache.org
> > >>> >
> > >>> >>>>>>> wrote:
> > >>> >>>>>>>
> > >>> >>>>>>> Interesting!
> > >>> >>>>>>>
> > >>> >>>>>>>> One thing I’m wondering about is
why you’ve added
> "-fullstack"
> > >>> to
> > >>> >>>>>>>> the
> > >>> >>>>>>>> artifactId and the hyracks module.
> > >>> >>>>>>>>
> > >>> >>>>>>>> Cheers,
> > >>> >>>>>>>> Till
> > >>> >>>>>>>>
> > >>> >>>>>>>>
> > >>> >>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon
wrote:
> > >>> >>>>>>>>
> > >>> >>>>>>>> I've gone ahead and tried merging
my topic branch with this
> > >>> change,
> > >>> >>>>>>>> and it
> > >>> >>>>>>>>
> > >>> >>>>>>>> turned out surprisingly well. I really
didn't have many
> > issues.
> > >>> I'll
> > >>> >>>>>>>>> summarize the process:
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> 1) Merge the change from asterixdb
with your topic branch
> > >>> checked
> > >>> >>>>>>>>> out, so
> > >>> >>>>>>>>> just 'git merge hyracks-merge2'.
> > >>> >>>>>>>>> The only real conflict should
be the pom, if you altered
> > that.
> > >>> I
> > >>> >>>>>>>>> found it
> > >>> >>>>>>>>> easiest to just replicate my changes
and take the upstream,
> > >>> rather
> > >>> >>>>>>>>> than
> > >>> >>>>>>>>> trying anything funny, since usually
pom changes are not
> > major.
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> 2) Add your hyracks folder as
a remote (for me, 'git remote
> > add
> > >>> >>>>>>>>> hyracks-local file:///home/...')
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> 3) Merge your hyracks topic branch
into asterixdb ( ' git
> > merge
> > >>> >>>>>>>>> hyracks-local/imaxon/hdfs')
> > >>> >>>>>>>>> This also worked pretty well,
the only extra hiccup besides
> > >>> the pom
> > >>> >>>>>>>>> was
> > >>> >>>>>>>>> files I had created. Those appeared
at the top level again
> > >>> after
> > >>> >>>>>>>>> the
> > >>> >>>>>>>>> merge.
> > >>> >>>>>>>>> But, all you have to do is move
them back down one folder
> > into
> > >>> >>>>>>>>> hyracks-fullstack.
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> That's about it really. I went
ahead and pushed this up to
> > >>> github
> > >>> >>>>>>>>> as
> > >>> >>>>>>>>> well
> > >>> >>>>>>>>> so if anyone would like to take
a look at the process or
> > check
> > >>> out
> > >>> >>>>>>>>> the
> > >>> >>>>>>>>> branch to see what happened (at
least for me), the branch
> is
> > >>> here:
> > >>> >>>>>>>>>
> > >>> >>>>>>>>>
> > >>> >>>>>>>>>
> > >>> >>>>>>>>>
> > >>>
> >
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> Thanks,
> > >>> >>>>>>>>> -Ian
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM,
Ian Maxon <imaxon@uci.edu
> >
> > >>> wrote:
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> Chris found an issue with the
way git histories were being
> > >>> handled
> > >>> >>>>>>>>> in
> > >>> >>>>>>>>> the
> > >>> >>>>>>>>>
> > >>> >>>>>>>>> way I merged things, so I have
revised the proposed branch:
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>>
> > >>>
> > https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>> Basically I was trying to
fit everything into one commit,
> > >>> because
> > >>> >>>>>>>>>> I
> > >>> >>>>>>>>>> thought at first that I could
submit it to Gerrit that
> way.
> > >>> >>>>>>>>>> However
> > >>> >>>>>>>>>> that
> > >>> >>>>>>>>>> doesn't work for other reasons,
basically Gerrit tries to
> > >>> treat
> > >>> >>>>>>>>>> every new
> > >>> >>>>>>>>>> commit from Hyracks as a new
change. Splitting the commits
> > of
> > >>> the
> > >>> >>>>>>>>>> repository merge fixes the
issue.
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>> @Till, I think that creating
a textual patch would just be
> > >>> more
> > >>> >>>>>>>>>> work. If
> > >>> >>>>>>>>>> I
> > >>> >>>>>>>>>> were to do it that way I would
try fetching the Gerrit
> > patch,
> > >>> and
> > >>> >>>>>>>>>> then
> > >>> >>>>>>>>>> cherry-picking it onto a new
branch that has the
> > >>> hyracks+asterix
> > >>> >>>>>>>>>> master
> > >>> >>>>>>>>>> as
> > >>> >>>>>>>>>> the head.
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>> On Wed, Mar 30, 2016 at 5:42
PM, Till Westmann <
> > >>> tillw@apache.org>
> > >>> >>>>>>>>>> wrote:
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>> To get existing patches in,
could we just create a textual
> > >>> patch
> > >>> >>>>>>>>>> (e.g.
> > >>> >>>>>>>>>>
> > >>> >>>>>>>>>> from gerrit), apply that with
the necessary -p option to a
> > new
> > >>> >>>>>>>>>>> local
> > >>> >>>>>>>>>>> checkout of the merged
repositories and submit a new
> review
> > >>> to
> > >>> >>>>>>>>>>> gerrit?
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>> Thanks,
> > >>> >>>>>>>>>>> Till
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>> On 30 Mar 2016, at 12:36,
Ian Maxon wrote:
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>> Hi all,
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>> I went ahead preliminarily
merged the Hyracks and
> AsterixDB
> > >>> >>>>>>>>>>>> repositories
> > >>> >>>>>>>>>>>> into one. Unfortunately
this can't be reviewed in Gerrit
> > so
> > >>> you
> > >>> >>>>>>>>>>>> all can
> > >>> >>>>>>>>>>>> check it out here:
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>
> > >>>
> >
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>> You will likely have to
do some ugly rebasing for
> whatever
> > >>> >>>>>>>>>>>> changes
> > >>> >>>>>>>>>>>> you
> > >>> >>>>>>>>>>>> might have open once
this gets done, since it moves
> > >>> asterixdb
> > >>> >>>>>>>>>>>> down
> > >>> >>>>>>>>>>>> one
> > >>> >>>>>>>>>>>> folder and swaps out
pom.xml in the repository root.
> > >>> Hyracks is
> > >>> >>>>>>>>>>>> in
> > >>> >>>>>>>>>>>> a
> > >>> >>>>>>>>>>>> similar situation,
though you would want to reapply your
> > >>> change
> > >>> >>>>>>>>>>>> to
> > >>> >>>>>>>>>>>> the
> > >>> >>>>>>>>>>>> AsterixDB repo from
Hyracks (which is a bit odd). If you
> > >>> would
> > >>> >>>>>>>>>>>> like to
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>> see
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>> how this affects your
branch please do try fetching the
> > >>> branch I
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>>> linked
> > >>> >>>>>>>>>>>> above and testing
it out on a copy of your topic branch.
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>> I'm still making sure
all of the tests pass but
> nothing's
> > >>> >>>>>>>>>>>> failed so
> > >>> >>>>>>>>>>>> far.
> > >>> >>>>>>>>>>>> Unless anyone has
objections I think we should push this
> > >>> change
> > >>> >>>>>>>>>>>> either
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>> this
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>> week or early next week.
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>>> Let me know what you
all think.
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>> Thanks,
> > >>> >>>>>>>>>>>> - Ian
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>>
> > >>> >>>>>>>>>>>
> > >>> >>>>>>>>>>
> > >>> >>
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message