asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Maxon <ima...@uci.edu>
Subject Re: The Great Merge
Date Thu, 07 Apr 2016 16:12:40 GMT
Also, please don't push any new changes to Hyracks. You will need to merge
these into your asterix change or a new change if it is a hyracks change
with no asterixdb content. I have disabled the jenkins job that verifies
hyracks patches on Gerrit so hopefully nothing gets through.

On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <imaxon@uci.edu> wrote:

> Alright, everything is pushed. Please post here or start another thread if
> you start to experience issues.
>
> On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <imaxon@uci.edu> wrote:
>
>> Just as a reminder, I'll be merging this tommorow since it seems like
>> everything should go fairly smoothly.
>>
>> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sjaco002@ucr.edu> wrote:
>>
>>> http://landbeforetime.wikia.com/wiki/Great_Valley
>>>
>>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sjaco002@ucr.edu> wrote:
>>>
>>> > No, we are living in the GREAT valley :)
>>> > Steven
>>> >
>>> > On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dtabass@gmail.com> wrote:
>>> >
>>> >> Sounds like things are GOOD!  Excellent.  (So not to be feared like
>>> the
>>> >> event that the name of this one keeps reminding me of:
>>> >> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
>>> >>
>>> >>
>>> >> On 4/4/16 1:12 PM, Steven Jacobs wrote:
>>> >>
>>> >>> It seems that I might be the only one concerned here, but it seems
>>> like
>>> >>> there should be others, so I am continuing this thread.
>>> >>>
>>> >>> I modified the perl REGEX from Chris' summer solution, and it works!
>>> >>>
>>> >>> Once Ian has merged master:
>>> >>>
>>> >>> 1. On your local branch, find the *parent* of the first commit you
>>> want
>>> >>> to
>>> >>> migrate onto the new master, e.g.
>>> >>> de6e0da24c26037967eb9a937d2c77c6c43e8761
>>> >>>
>>> >>> 2. Run this magic command:
>>> >>>
>>> >>>     git format-patch --stdout
>>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
>>> >>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
>>> >>>
>>> >>> 3. Now fetch master, and create a new local branch from it:
>>> >>>
>>> >>>     git switch master; git pull; git checkout -B newbranch
>>> >>>
>>> >>> 4. Apply your tweaked patch:
>>> >>>
>>> >>>     git am /tmp/my.patch
>>> >>>
>>> >>>
>>> >>> This recognized ALL of my file moves/renames and applied them
>>> correctly.
>>> >>> It
>>> >>> leaves only two issues:
>>> >>> 1) Something similar will probably need to be done for Hyracks
>>> changes
>>> >>> 2) My pom changes didn't apply. This isn't so bad since there are
>>> only a
>>> >>> few pom files total.
>>> >>>
>>> >>>
>>> >>> I hope this helps,
>>> >>> Steven
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sjaco002@ucr.edu>
>>> wrote:
>>> >>>
>>> >>> Here is Chris's original solution to give context. I think changing
>>> the
>>> >>>> REGEX might be enough to re-use the solution:
>>> >>>>
>>> >>>> 1. On your local branch, find the *parent* of the first commit
you
>>> want
>>> >>>> to
>>> >>>> migrate onto the new master. If you were fully up-to-date before
the
>>> >>>> repackaging commits went in, this will be Till's
>>> >>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use
that
>>> here.
>>> >>>>
>>> >>>> 2. Run this magic command:
>>> >>>>
>>> >>>>     git format-patch --stdout
>>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
>>> >>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
>>> >>>>
>>> >>>> 3. Now fetch the new master, and create a new local branch from
it:
>>> >>>>
>>> >>>>     git switch master; git pull; git checkout -B newbranch
>>> >>>>
>>> >>>> 4. Apply your tweaked patch:
>>> >>>>
>>> >>>>     git am /tmp/my.patch
>>> >>>>
>>> >>>>
>>> >>>> Steven
>>> >>>>
>>> >>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sjaco002@ucr.edu>
>>> >>>> wrote:
>>> >>>>
>>> >>>> I've tried doing this now on my branch.
>>> >>>>> As I feared, all of the files that are renamed/moved become
>>> conflicts
>>> >>>>> (just a few hundred conflicts in my case 😑).
>>> >>>>> I'm wondering if we could use a similar technique for what
we did
>>> >>>>> during
>>> >>>>> the summer (for the apache change) to get around this.
>>> >>>>>
>>> >>>>> Steven
>>> >>>>>
>>> >>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <tillw@apache.org>
>>> >>>>> wrote:
>>> >>>>>
>>> >>>>> I’m not sure I completely understand what you are saying.
Is this a
>>> >>>>>> temporary state that will get cleaned up later or is
this
>>> supposed to
>>> >>>>>> stay this way (having "-fullstack" in the names)?
>>> >>>>>>
>>> >>>>>> Thanks,
>>> >>>>>> Till
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>>> >>>>>>
>>> >>>>>> I'm not sure if it was necessary to rename it, but the
original
>>> issue
>>> >>>>>> is
>>> >>>>>>
>>> >>>>>>> that the hyracks repo itself has a folder named
hyracks, that
>>> >>>>>>> contains
>>> >>>>>>> hyracks. I thought this might confuse git if I did
something like
>>> >>>>>>> make a
>>> >>>>>>> new temporary folder, move everything into that,
and then rename
>>> it
>>> >>>>>>> to
>>> >>>>>>> 'hyracks'.
>>> >>>>>>>
>>> >>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <tillw@apache.org
>>> >
>>> >>>>>>> wrote:
>>> >>>>>>>
>>> >>>>>>> Interesting!
>>> >>>>>>>
>>> >>>>>>>> One thing I’m wondering about is why you’ve
added "-fullstack"
>>> to
>>> >>>>>>>> the
>>> >>>>>>>> artifactId and the hyracks module.
>>> >>>>>>>>
>>> >>>>>>>> Cheers,
>>> >>>>>>>> Till
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>> >>>>>>>>
>>> >>>>>>>> I've gone ahead and tried merging my topic branch
with this
>>> change,
>>> >>>>>>>> and it
>>> >>>>>>>>
>>> >>>>>>>> turned out surprisingly well. I really didn't
have many issues.
>>> I'll
>>> >>>>>>>>> summarize the process:
>>> >>>>>>>>>
>>> >>>>>>>>> 1) Merge the change from asterixdb with
your topic branch
>>> checked
>>> >>>>>>>>> out, so
>>> >>>>>>>>> just 'git merge hyracks-merge2'.
>>> >>>>>>>>> The only real conflict should be the pom,
if you altered that.
>>> I
>>> >>>>>>>>> found it
>>> >>>>>>>>> easiest to just replicate my changes and
take the upstream,
>>> rather
>>> >>>>>>>>> than
>>> >>>>>>>>> trying anything funny, since usually pom
changes are not major.
>>> >>>>>>>>>
>>> >>>>>>>>> 2) Add your hyracks folder as a remote (for
me, 'git remote add
>>> >>>>>>>>> hyracks-local file:///home/...')
>>> >>>>>>>>>
>>> >>>>>>>>> 3) Merge your hyracks topic branch into
asterixdb ( ' git merge
>>> >>>>>>>>> hyracks-local/imaxon/hdfs')
>>> >>>>>>>>> This also worked pretty well, the only extra
hiccup besides
>>> the pom
>>> >>>>>>>>> was
>>> >>>>>>>>> files I had created. Those appeared at the
top level again
>>> after
>>> >>>>>>>>> the
>>> >>>>>>>>> merge.
>>> >>>>>>>>> But, all you have to do is move them back
down one folder into
>>> >>>>>>>>> hyracks-fullstack.
>>> >>>>>>>>>
>>> >>>>>>>>> That's about it really. I went ahead and
pushed this up to
>>> github
>>> >>>>>>>>> as
>>> >>>>>>>>> well
>>> >>>>>>>>> so if anyone would like to take a look at
the process or check
>>> out
>>> >>>>>>>>> the
>>> >>>>>>>>> branch to see what happened (at least for
me), the branch is
>>> here:
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>> >>>>>>>>>
>>> >>>>>>>>> Thanks,
>>> >>>>>>>>> -Ian
>>> >>>>>>>>>
>>> >>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon
<imaxon@uci.edu>
>>> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>> Chris found an issue with the way git histories
were being
>>> handled
>>> >>>>>>>>> in
>>> >>>>>>>>> the
>>> >>>>>>>>>
>>> >>>>>>>>> way I merged things, so I have revised the
proposed branch:
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>> >>>>>>>>>>
>>> >>>>>>>>>> Basically I was trying to fit everything
into one commit,
>>> because
>>> >>>>>>>>>> I
>>> >>>>>>>>>> thought at first that I could submit
it to Gerrit that way.
>>> >>>>>>>>>> However
>>> >>>>>>>>>> that
>>> >>>>>>>>>> doesn't work for other reasons, basically
Gerrit tries to
>>> treat
>>> >>>>>>>>>> every new
>>> >>>>>>>>>> commit from Hyracks as a new change.
Splitting the commits of
>>> the
>>> >>>>>>>>>> repository merge fixes the issue.
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> @Till, I think that creating a textual
patch would just be
>>> more
>>> >>>>>>>>>> work. If
>>> >>>>>>>>>> I
>>> >>>>>>>>>> were to do it that way I would try fetching
the Gerrit patch,
>>> and
>>> >>>>>>>>>> then
>>> >>>>>>>>>> cherry-picking it onto a new branch
that has the
>>> hyracks+asterix
>>> >>>>>>>>>> master
>>> >>>>>>>>>> as
>>> >>>>>>>>>> the head.
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till
Westmann <
>>> tillw@apache.org>
>>> >>>>>>>>>> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>> To get existing patches in, could we
just create a textual
>>> patch
>>> >>>>>>>>>> (e.g.
>>> >>>>>>>>>>
>>> >>>>>>>>>> from gerrit), apply that with the necessary
-p option to a new
>>> >>>>>>>>>>> local
>>> >>>>>>>>>>> checkout of the merged repositories
and submit a new review
>>> to
>>> >>>>>>>>>>> gerrit?
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Thanks,
>>> >>>>>>>>>>> Till
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon
wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Hi all,
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> I went ahead preliminarily merged
the Hyracks and AsterixDB
>>> >>>>>>>>>>>> repositories
>>> >>>>>>>>>>>> into one. Unfortunately this
can't be reviewed in Gerrit so
>>> you
>>> >>>>>>>>>>>> all can
>>> >>>>>>>>>>>> check it out here:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> You will likely have to do some
ugly rebasing for whatever
>>> >>>>>>>>>>>> changes
>>> >>>>>>>>>>>> you
>>> >>>>>>>>>>>> might have open once this gets
done, since it moves
>>> asterixdb
>>> >>>>>>>>>>>> down
>>> >>>>>>>>>>>> one
>>> >>>>>>>>>>>> folder and swaps out pom.xml
in the repository root.
>>> Hyracks is
>>> >>>>>>>>>>>> in
>>> >>>>>>>>>>>> a
>>> >>>>>>>>>>>> similar situation, though you
would want to reapply your
>>> change
>>> >>>>>>>>>>>> to
>>> >>>>>>>>>>>> the
>>> >>>>>>>>>>>> AsterixDB repo from Hyracks
(which is a bit odd). If you
>>> would
>>> >>>>>>>>>>>> like to
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> see
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>> how this affects your branch please
do try fetching the
>>> branch I
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> linked
>>> >>>>>>>>>>>> above and testing it out on
a copy of your topic branch.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I'm still making sure all of
the tests pass but nothing's
>>> >>>>>>>>>>>> failed so
>>> >>>>>>>>>>>> far.
>>> >>>>>>>>>>>> Unless anyone has objections
I think we should push this
>>> change
>>> >>>>>>>>>>>> either
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> this
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>> week or early next week.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> Let me know what you all think.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>> - Ian
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>
>>> >
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message