asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 李文海 <...@whu.edu.cn>
Subject Re: Re: The Great Merge
Date Fri, 08 Apr 2016 08:48:17 GMT
Cool, really great.


> -----Original Messages-----
> From: "Ian Maxon" <imaxon@uci.edu>
> Sent Time: Friday, April 8, 2016
> To: dev@asterixdb.incubator.apache.org
> Cc: 
> Subject: Re: The Great Merge
> 
> Say if you had an AsterixDB change in a topic branch that had both
> AsterixDB and Hyracks components, it'd be like:
> 
> git checkout (topic branch)
> git fetch origin
> git merge origin/master
> (resolve conflicts)
> git commit
> git remote add hyracks-local file://(absolute path to your hyracks repo)
> git fetch hyracks-local
> git merge hyracks-local/(name of your hyracks topic branch)
> (resolve conflicts)
> 
> 
> On Thu, Apr 7, 2016 at 7:31 PM, Taewoo Kim <wangsaeu@gmail.com> wrote:
> 
> > @Ian:
> >
> > Could you show the process (the explicit command, if possible) step by
> > step? I know you did. But, I have a hard time to apply your merge. Thank
> > you. Steven's patch should work, also. But it didn't work on my branch. It
> > looks like I miss something.
> >
> >
> >
> > Best,
> > Taewoo
> >
> > On Thu, Apr 7, 2016 at 9:12 AM, Ian Maxon <imaxon@uci.edu> wrote:
> >
> > > Also, please don't push any new changes to Hyracks. You will need to
> > merge
> > > these into your asterix change or a new change if it is a hyracks change
> > > with no asterixdb content. I have disabled the jenkins job that verifies
> > > hyracks patches on Gerrit so hopefully nothing gets through.
> > >
> > > On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <imaxon@uci.edu> wrote:
> > >
> > > > Alright, everything is pushed. Please post here or start another thread
> > > if
> > > > you start to experience issues.
> > > >
> > > > On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <imaxon@uci.edu> wrote:
> > > >
> > > >> Just as a reminder, I'll be merging this tommorow since it seems like
> > > >> everything should go fairly smoothly.
> > > >>
> > > >> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sjaco002@ucr.edu>
> > wrote:
> > > >>
> > > >>> http://landbeforetime.wikia.com/wiki/Great_Valley
> > > >>>
> > > >>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sjaco002@ucr.edu>
> > > wrote:
> > > >>>
> > > >>> > No, we are living in the GREAT valley :)
> > > >>> > Steven
> > > >>> >
> > > >>> > On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dtabass@gmail.com>
> > > wrote:
> > > >>> >
> > > >>> >> Sounds like things are GOOD!  Excellent.  (So not to
be feared
> > like
> > > >>> the
> > > >>> >> event that the name of this one keeps reminding me of:
> > > >>> >> http://landbeforetime.wikia.com/wiki/Great_Earthshake
:-).)
> > > >>> >>
> > > >>> >>
> > > >>> >> On 4/4/16 1:12 PM, Steven Jacobs wrote:
> > > >>> >>
> > > >>> >>> It seems that I might be the only one concerned here,
but it
> > seems
> > > >>> like
> > > >>> >>> there should be others, so I am continuing this thread.
> > > >>> >>>
> > > >>> >>> I modified the perl REGEX from Chris' summer solution,
and it
> > > works!
> > > >>> >>>
> > > >>> >>> Once Ian has merged master:
> > > >>> >>>
> > > >>> >>> 1. On your local branch, find the *parent* of the
first commit
> > you
> > > >>> want
> > > >>> >>> to
> > > >>> >>> migrate onto the new master, e.g.
> > > >>> >>> de6e0da24c26037967eb9a937d2c77c6c43e8761
> > > >>> >>>
> > > >>> >>> 2. Run this magic command:
> > > >>> >>>
> > > >>> >>>     git format-patch --stdout
> > > >>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
> > > >>> >>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
> > > >>> >>>
> > > >>> >>> 3. Now fetch master, and create a new local branch
from it:
> > > >>> >>>
> > > >>> >>>     git switch master; git pull; git checkout -B
newbranch
> > > >>> >>>
> > > >>> >>> 4. Apply your tweaked patch:
> > > >>> >>>
> > > >>> >>>     git am /tmp/my.patch
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> This recognized ALL of my file moves/renames and
applied them
> > > >>> correctly.
> > > >>> >>> It
> > > >>> >>> leaves only two issues:
> > > >>> >>> 1) Something similar will probably need to be done
for Hyracks
> > > >>> changes
> > > >>> >>> 2) My pom changes didn't apply. This isn't so bad
since there are
> > > >>> only a
> > > >>> >>> few pom files total.
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> I hope this helps,
> > > >>> >>> Steven
> > > >>> >>>
> > > >>> >>>
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sjaco002@ucr.edu
> > >
> > > >>> wrote:
> > > >>> >>>
> > > >>> >>> Here is Chris's original solution to give context.
I think
> > changing
> > > >>> the
> > > >>> >>>> REGEX might be enough to re-use the solution:
> > > >>> >>>>
> > > >>> >>>> 1. On your local branch, find the *parent* of
the first commit
> > you
> > > >>> want
> > > >>> >>>> to
> > > >>> >>>> migrate onto the new master. If you were fully
up-to-date before
> > > the
> > > >>> >>>> repackaging commits went in, this will be Till's
> > > >>> >>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53,
so I'll use
> > that
> > > >>> here.
> > > >>> >>>>
> > > >>> >>>> 2. Run this magic command:
> > > >>> >>>>
> > > >>> >>>>     git format-patch --stdout
> > > >>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
> > > >>> >>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' >
/tmp/my.patch
> > > >>> >>>>
> > > >>> >>>> 3. Now fetch the new master, and create a new
local branch from
> > > it:
> > > >>> >>>>
> > > >>> >>>>     git switch master; git pull; git checkout
-B newbranch
> > > >>> >>>>
> > > >>> >>>> 4. Apply your tweaked patch:
> > > >>> >>>>
> > > >>> >>>>     git am /tmp/my.patch
> > > >>> >>>>
> > > >>> >>>>
> > > >>> >>>> Steven
> > > >>> >>>>
> > > >>> >>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs
<
> > sjaco002@ucr.edu>
> > > >>> >>>> wrote:
> > > >>> >>>>
> > > >>> >>>> I've tried doing this now on my branch.
> > > >>> >>>>> As I feared, all of the files that are renamed/moved
become
> > > >>> conflicts
> > > >>> >>>>> (just a few hundred conflicts in my case
😑).
> > > >>> >>>>> I'm wondering if we could use a similar technique
for what we
> > did
> > > >>> >>>>> during
> > > >>> >>>>> the summer (for the apache change) to get
around this.
> > > >>> >>>>>
> > > >>> >>>>> Steven
> > > >>> >>>>>
> > > >>> >>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann
<
> > tillw@apache.org>
> > > >>> >>>>> wrote:
> > > >>> >>>>>
> > > >>> >>>>> I’m not sure I completely understand what
you are saying. Is
> > > this a
> > > >>> >>>>>> temporary state that will get cleaned
up later or is this
> > > >>> supposed to
> > > >>> >>>>>> stay this way (having "-fullstack" in
the names)?
> > > >>> >>>>>>
> > > >>> >>>>>> Thanks,
> > > >>> >>>>>> Till
> > > >>> >>>>>>
> > > >>> >>>>>>
> > > >>> >>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
> > > >>> >>>>>>
> > > >>> >>>>>> I'm not sure if it was necessary to rename
it, but the
> > original
> > > >>> issue
> > > >>> >>>>>> is
> > > >>> >>>>>>
> > > >>> >>>>>>> that the hyracks repo itself has
a folder named hyracks, that
> > > >>> >>>>>>> contains
> > > >>> >>>>>>> hyracks. I thought this might confuse
git if I did something
> > > like
> > > >>> >>>>>>> make a
> > > >>> >>>>>>> new temporary folder, move everything
into that, and then
> > > rename
> > > >>> it
> > > >>> >>>>>>> to
> > > >>> >>>>>>> 'hyracks'.
> > > >>> >>>>>>>
> > > >>> >>>>>>> On Thu, Mar 31, 2016 at 6:35 PM,
Till Westmann <
> > > tillw@apache.org
> > > >>> >
> > > >>> >>>>>>> wrote:
> > > >>> >>>>>>>
> > > >>> >>>>>>> Interesting!
> > > >>> >>>>>>>
> > > >>> >>>>>>>> One thing I’m wondering about
is why you’ve added
> > "-fullstack"
> > > >>> to
> > > >>> >>>>>>>> the
> > > >>> >>>>>>>> artifactId and the hyracks module.
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> Cheers,
> > > >>> >>>>>>>> Till
> > > >>> >>>>>>>>
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> On 31 Mar 2016, at 17:21, Ian
Maxon wrote:
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> I've gone ahead and tried merging
my topic branch with this
> > > >>> change,
> > > >>> >>>>>>>> and it
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> turned out surprisingly well.
I really didn't have many
> > > issues.
> > > >>> I'll
> > > >>> >>>>>>>>> summarize the process:
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> 1) Merge the change from
asterixdb with your topic branch
> > > >>> checked
> > > >>> >>>>>>>>> out, so
> > > >>> >>>>>>>>> just 'git merge hyracks-merge2'.
> > > >>> >>>>>>>>> The only real conflict should
be the pom, if you altered
> > > that.
> > > >>> I
> > > >>> >>>>>>>>> found it
> > > >>> >>>>>>>>> easiest to just replicate
my changes and take the upstream,
> > > >>> rather
> > > >>> >>>>>>>>> than
> > > >>> >>>>>>>>> trying anything funny, since
usually pom changes are not
> > > major.
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> 2) Add your hyracks folder
as a remote (for me, 'git remote
> > > add
> > > >>> >>>>>>>>> hyracks-local file:///home/...')
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> 3) Merge your hyracks topic
branch into asterixdb ( ' git
> > > merge
> > > >>> >>>>>>>>> hyracks-local/imaxon/hdfs')
> > > >>> >>>>>>>>> This also worked pretty well,
the only extra hiccup besides
> > > >>> the pom
> > > >>> >>>>>>>>> was
> > > >>> >>>>>>>>> files I had created. Those
appeared at the top level again
> > > >>> after
> > > >>> >>>>>>>>> the
> > > >>> >>>>>>>>> merge.
> > > >>> >>>>>>>>> But, all you have to do is
move them back down one folder
> > > into
> > > >>> >>>>>>>>> hyracks-fullstack.
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> That's about it really. I
went ahead and pushed this up to
> > > >>> github
> > > >>> >>>>>>>>> as
> > > >>> >>>>>>>>> well
> > > >>> >>>>>>>>> so if anyone would like to
take a look at the process or
> > > check
> > > >>> out
> > > >>> >>>>>>>>> the
> > > >>> >>>>>>>>> branch to see what happened
(at least for me), the branch
> > is
> > > >>> here:
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>>
> > > >>>
> > >
> > https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> Thanks,
> > > >>> >>>>>>>>> -Ian
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> On Wed, Mar 30, 2016 at 6:17
PM, Ian Maxon <imaxon@uci.edu
> > >
> > > >>> wrote:
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> Chris found an issue with
the way git histories were being
> > > >>> handled
> > > >>> >>>>>>>>> in
> > > >>> >>>>>>>>> the
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> way I merged things, so I
have revised the proposed branch:
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>>
> > > https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> Basically I was trying
to fit everything into one commit,
> > > >>> because
> > > >>> >>>>>>>>>> I
> > > >>> >>>>>>>>>> thought at first that
I could submit it to Gerrit that
> > way.
> > > >>> >>>>>>>>>> However
> > > >>> >>>>>>>>>> that
> > > >>> >>>>>>>>>> doesn't work for other
reasons, basically Gerrit tries to
> > > >>> treat
> > > >>> >>>>>>>>>> every new
> > > >>> >>>>>>>>>> commit from Hyracks as
a new change. Splitting the commits
> > > of
> > > >>> the
> > > >>> >>>>>>>>>> repository merge fixes
the issue.
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> @Till, I think that creating
a textual patch would just be
> > > >>> more
> > > >>> >>>>>>>>>> work. If
> > > >>> >>>>>>>>>> I
> > > >>> >>>>>>>>>> were to do it that way
I would try fetching the Gerrit
> > > patch,
> > > >>> and
> > > >>> >>>>>>>>>> then
> > > >>> >>>>>>>>>> cherry-picking it onto
a new branch that has the
> > > >>> hyracks+asterix
> > > >>> >>>>>>>>>> master
> > > >>> >>>>>>>>>> as
> > > >>> >>>>>>>>>> the head.
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> On Wed, Mar 30, 2016
at 5:42 PM, Till Westmann <
> > > >>> tillw@apache.org>
> > > >>> >>>>>>>>>> wrote:
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> To get existing patches
in, could we just create a textual
> > > >>> patch
> > > >>> >>>>>>>>>> (e.g.
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> from gerrit), apply that
with the necessary -p option to a
> > > new
> > > >>> >>>>>>>>>>> local
> > > >>> >>>>>>>>>>> checkout of the merged
repositories and submit a new
> > review
> > > >>> to
> > > >>> >>>>>>>>>>> gerrit?
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> Thanks,
> > > >>> >>>>>>>>>>> Till
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> On 30 Mar 2016, at
12:36, Ian Maxon wrote:
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> Hi all,
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> I went ahead preliminarily
merged the Hyracks and
> > AsterixDB
> > > >>> >>>>>>>>>>>> repositories
> > > >>> >>>>>>>>>>>> into one. Unfortunately
this can't be reviewed in Gerrit
> > > so
> > > >>> you
> > > >>> >>>>>>>>>>>> all can
> > > >>> >>>>>>>>>>>> check it out
here:
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>
> > > >>>
> > >
> > https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> You will likely have
to do some ugly rebasing for
> > whatever
> > > >>> >>>>>>>>>>>> changes
> > > >>> >>>>>>>>>>>> you
> > > >>> >>>>>>>>>>>> might have open
once this gets done, since it moves
> > > >>> asterixdb
> > > >>> >>>>>>>>>>>> down
> > > >>> >>>>>>>>>>>> one
> > > >>> >>>>>>>>>>>> folder and swaps
out pom.xml in the repository root.
> > > >>> Hyracks is
> > > >>> >>>>>>>>>>>> in
> > > >>> >>>>>>>>>>>> a
> > > >>> >>>>>>>>>>>> similar situation,
though you would want to reapply your
> > > >>> change
> > > >>> >>>>>>>>>>>> to
> > > >>> >>>>>>>>>>>> the
> > > >>> >>>>>>>>>>>> AsterixDB repo
from Hyracks (which is a bit odd). If you
> > > >>> would
> > > >>> >>>>>>>>>>>> like to
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> see
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>> how this affects
your branch please do try fetching the
> > > >>> branch I
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>>> linked
> > > >>> >>>>>>>>>>>> above and testing
it out on a copy of your topic branch.
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> I'm still making
sure all of the tests pass but
> > nothing's
> > > >>> >>>>>>>>>>>> failed so
> > > >>> >>>>>>>>>>>> far.
> > > >>> >>>>>>>>>>>> Unless anyone
has objections I think we should push this
> > > >>> change
> > > >>> >>>>>>>>>>>> either
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> this
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>> week or early next
week.
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>>> Let me know what
you all think.
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> Thanks,
> > > >>> >>>>>>>>>>>> - Ian
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>> >>
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >


Mime
View raw message