asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Westmann" <ti...@apache.org>
Subject [WARNING] Don't push anything into the hyracks git repository (was: The Great Merge)
Date Thu, 07 Apr 2016 17:24:35 GMT
Jut a forward for people (like myself) that don’t always see which 
messages are important :)

Cheers,
Till

Forwarded message:

> From: Ian Maxon <imaxon@uci.edu>
> To: dev@asterixdb.incubator.apache.org
> Subject: Re: The Great Merge
> Date: Thu, 7 Apr 2016 09:12:40 -0700
>
> Also, please don't push any new changes to Hyracks. You will need to 
> merge
> these into your asterix change or a new change if it is a hyracks 
> change
> with no asterixdb content. I have disabled the jenkins job that 
> verifies
> hyracks patches on Gerrit so hopefully nothing gets through.
>
> On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <imaxon@uci.edu> wrote:
>
>> Alright, everything is pushed. Please post here or start another 
>> thread if
>> you start to experience issues.
>>
>> On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <imaxon@uci.edu> wrote:
>>
>>> Just as a reminder, I'll be merging this tommorow since it seems 
>>> like
>>> everything should go fairly smoothly.
>>>
>>> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sjaco002@ucr.edu> 
>>> wrote:
>>>
>>>> http://landbeforetime.wikia.com/wiki/Great_Valley
>>>>
>>>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sjaco002@ucr.edu> 
>>>> wrote:
>>>>
>>>>> No, we are living in the GREAT valley :)
>>>>> Steven
>>>>>
>>>>> On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dtabass@gmail.com>

>>>>> wrote:
>>>>>
>>>>>> Sounds like things are GOOD!  Excellent.  (So not to be feared 
>>>>>> like
>>>> the
>>>>>> event that the name of this one keeps reminding me of:
>>>>>> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
>>>>>>
>>>>>>
>>>>>> On 4/4/16 1:12 PM, Steven Jacobs wrote:
>>>>>>
>>>>>>> It seems that I might be the only one concerned here, but it

>>>>>>> seems
>>>> like
>>>>>>> there should be others, so I am continuing this thread.
>>>>>>>
>>>>>>> I modified the perl REGEX from Chris' summer solution, and it

>>>>>>> works!
>>>>>>>
>>>>>>> Once Ian has merged master:
>>>>>>>
>>>>>>> 1. On your local branch, find the *parent* of the first commit

>>>>>>> you
>>>> want
>>>>>>> to
>>>>>>> migrate onto the new master, e.g.
>>>>>>> de6e0da24c26037967eb9a937d2c77c6c43e8761
>>>>>>>
>>>>>>> 2. Run this magic command:
>>>>>>>
>>>>>>>     git format-patch --stdout
>>>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
>>>>>>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
>>>>>>>
>>>>>>> 3. Now fetch master, and create a new local branch from it:
>>>>>>>
>>>>>>>     git switch master; git pull; git checkout -B newbranch
>>>>>>>
>>>>>>> 4. Apply your tweaked patch:
>>>>>>>
>>>>>>>     git am /tmp/my.patch
>>>>>>>
>>>>>>>
>>>>>>> This recognized ALL of my file moves/renames and applied them
>>>> correctly.
>>>>>>> It
>>>>>>> leaves only two issues:
>>>>>>> 1) Something similar will probably need to be done for Hyracks
>>>> changes
>>>>>>> 2) My pom changes didn't apply. This isn't so bad since there

>>>>>>> are
>>>> only a
>>>>>>> few pom files total.
>>>>>>>
>>>>>>>
>>>>>>> I hope this helps,
>>>>>>> Steven
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs 
>>>>>>> <sjaco002@ucr.edu>
>>>> wrote:
>>>>>>>
>>>>>>> Here is Chris's original solution to give context. I think 
>>>>>>> changing
>>>> the
>>>>>>>> REGEX might be enough to re-use the solution:
>>>>>>>>
>>>>>>>> 1. On your local branch, find the *parent* of the first commit

>>>>>>>> you
>>>> want
>>>>>>>> to
>>>>>>>> migrate onto the new master. If you were fully up-to-date

>>>>>>>> before the
>>>>>>>> repackaging commits went in, this will be Till's
>>>>>>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll
use 
>>>>>>>> that
>>>> here.
>>>>>>>>
>>>>>>>> 2. Run this magic command:
>>>>>>>>
>>>>>>>>     git format-patch --stdout
>>>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
>>>>>>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
>>>>>>>>
>>>>>>>> 3. Now fetch the new master, and create a new local branch
from 
>>>>>>>> it:
>>>>>>>>
>>>>>>>>     git switch master; git pull; git checkout -B newbranch
>>>>>>>>
>>>>>>>> 4. Apply your tweaked patch:
>>>>>>>>
>>>>>>>>     git am /tmp/my.patch
>>>>>>>>
>>>>>>>>
>>>>>>>> Steven
>>>>>>>>
>>>>>>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs 
>>>>>>>> <sjaco002@ucr.edu>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> I've tried doing this now on my branch.
>>>>>>>>> As I feared, all of the files that are renamed/moved
become
>>>> conflicts
>>>>>>>>> (just a few hundred conflicts in my case 😑).
>>>>>>>>> I'm wondering if we could use a similar technique for
what we 
>>>>>>>>> did
>>>>>>>>> during
>>>>>>>>> the summer (for the apache change) to get around this.
>>>>>>>>>
>>>>>>>>> Steven
>>>>>>>>>
>>>>>>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann 
>>>>>>>>> <tillw@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I’m not sure I completely understand what you are saying.
Is 
>>>>>>>>> this a
>>>>>>>>>> temporary state that will get cleaned up later or
is this
>>>> supposed to
>>>>>>>>>> stay this way (having "-fullstack" in the names)?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Till
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>>>>>>>>>>
>>>>>>>>>> I'm not sure if it was necessary to rename it, but
the 
>>>>>>>>>> original
>>>> issue
>>>>>>>>>> is
>>>>>>>>>>
>>>>>>>>>>> that the hyracks repo itself has a folder named
hyracks, 
>>>>>>>>>>> that
>>>>>>>>>>> contains
>>>>>>>>>>> hyracks. I thought this might confuse git if
I did something 
>>>>>>>>>>> like
>>>>>>>>>>> make a
>>>>>>>>>>> new temporary folder, move everything into that,
and then 
>>>>>>>>>>> rename
>>>> it
>>>>>>>>>>> to
>>>>>>>>>>> 'hyracks'.
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann

>>>>>>>>>>> <tillw@apache.org
>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Interesting!
>>>>>>>>>>>
>>>>>>>>>>>> One thing I’m wondering about is why you’ve
added 
>>>>>>>>>>>> "-fullstack"
>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>> artifactId and the hyracks module.
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>> Till
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I've gone ahead and tried merging my topic
branch with this
>>>> change,
>>>>>>>>>>>> and it
>>>>>>>>>>>>
>>>>>>>>>>>> turned out surprisingly well. I really didn't
have many 
>>>>>>>>>>>> issues.
>>>> I'll
>>>>>>>>>>>>> summarize the process:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) Merge the change from asterixdb with
your topic branch
>>>> checked
>>>>>>>>>>>>> out, so
>>>>>>>>>>>>> just 'git merge hyracks-merge2'.
>>>>>>>>>>>>> The only real conflict should be the
pom, if you altered 
>>>>>>>>>>>>> that.
>>>> I
>>>>>>>>>>>>> found it
>>>>>>>>>>>>> easiest to just replicate my changes
and take the 
>>>>>>>>>>>>> upstream,
>>>> rather
>>>>>>>>>>>>> than
>>>>>>>>>>>>> trying anything funny, since usually
pom changes are not 
>>>>>>>>>>>>> major.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2) Add your hyracks folder as a remote
(for me, 'git 
>>>>>>>>>>>>> remote add
>>>>>>>>>>>>> hyracks-local file:///home/...')
>>>>>>>>>>>>>
>>>>>>>>>>>>> 3) Merge your hyracks topic branch into
asterixdb ( ' git 
>>>>>>>>>>>>> merge
>>>>>>>>>>>>> hyracks-local/imaxon/hdfs')
>>>>>>>>>>>>> This also worked pretty well, the only
extra hiccup 
>>>>>>>>>>>>> besides
>>>> the pom
>>>>>>>>>>>>> was
>>>>>>>>>>>>> files I had created. Those appeared at
the top level again
>>>> after
>>>>>>>>>>>>> the
>>>>>>>>>>>>> merge.
>>>>>>>>>>>>> But, all you have to do is move them
back down one folder 
>>>>>>>>>>>>> into
>>>>>>>>>>>>> hyracks-fullstack.
>>>>>>>>>>>>>
>>>>>>>>>>>>> That's about it really. I went ahead
and pushed this up to
>>>> github
>>>>>>>>>>>>> as
>>>>>>>>>>>>> well
>>>>>>>>>>>>> so if anyone would like to take a look
at the process or 
>>>>>>>>>>>>> check
>>>> out
>>>>>>>>>>>>> the
>>>>>>>>>>>>> branch to see what happened (at least
for me), the branch 
>>>>>>>>>>>>> is
>>>> here:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -Ian
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian
Maxon 
>>>>>>>>>>>>> <imaxon@uci.edu>
>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Chris found an issue with the way git
histories were being
>>>> handled
>>>>>>>>>>>>> in
>>>>>>>>>>>>> the
>>>>>>>>>>>>>
>>>>>>>>>>>>> way I merged things, so I have revised
the proposed 
>>>>>>>>>>>>> branch:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Basically I was trying to fit everything
into one commit,
>>>> because
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>> thought at first that I could submit
it to Gerrit that 
>>>>>>>>>>>>>> way.
>>>>>>>>>>>>>> However
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> doesn't work for other reasons, basically
Gerrit tries to
>>>> treat
>>>>>>>>>>>>>> every new
>>>>>>>>>>>>>> commit from Hyracks as a new change.
Splitting the 
>>>>>>>>>>>>>> commits of
>>>> the
>>>>>>>>>>>>>> repository merge fixes the issue.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> @Till, I think that creating a textual
patch would just 
>>>>>>>>>>>>>> be
>>>> more
>>>>>>>>>>>>>> work. If
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>> were to do it that way I would try
fetching the Gerrit 
>>>>>>>>>>>>>> patch,
>>>> and
>>>>>>>>>>>>>> then
>>>>>>>>>>>>>> cherry-picking it onto a new branch
that has the
>>>> hyracks+asterix
>>>>>>>>>>>>>> master
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>> the head.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM,
Till Westmann <
>>>> tillw@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To get existing patches in, could
we just create a 
>>>>>>>>>>>>>> textual
>>>> patch
>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> from gerrit), apply that with the
necessary -p option to 
>>>>>>>>>>>>>> a new
>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>> checkout of the merged repositories
and submit a new 
>>>>>>>>>>>>>>> review
>>>> to
>>>>>>>>>>>>>>> gerrit?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Till
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian
Maxon wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I went ahead preliminarily merged
the Hyracks and 
>>>>>>>>>>>>>>> AsterixDB
>>>>>>>>>>>>>>>> repositories
>>>>>>>>>>>>>>>> into one. Unfortunately this
can't be reviewed in 
>>>>>>>>>>>>>>>> Gerrit so
>>>> you
>>>>>>>>>>>>>>>> all can
>>>>>>>>>>>>>>>> check it out here:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You will likely have to do some
ugly rebasing for 
>>>>>>>>>>>>>>> whatever
>>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>> might have open once this
gets done, since it moves
>>>> asterixdb
>>>>>>>>>>>>>>>> down
>>>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>> folder and swaps out pom.xml
in the repository root.
>>>> Hyracks is
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> similar situation, though
you would want to reapply 
>>>>>>>>>>>>>>>> your
>>>> change
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> AsterixDB repo from Hyracks
(which is a bit odd). If 
>>>>>>>>>>>>>>>> you
>>>> would
>>>>>>>>>>>>>>>> like to
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> how this affects your branch
please do try fetching the
>>>> branch I
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> linked
>>>>>>>>>>>>>>>> above and testing it out
on a copy of your topic 
>>>>>>>>>>>>>>>> branch.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm still making sure all
of the tests pass but 
>>>>>>>>>>>>>>>> nothing's
>>>>>>>>>>>>>>>> failed so
>>>>>>>>>>>>>>>> far.
>>>>>>>>>>>>>>>> Unless anyone has objections
I think we should push 
>>>>>>>>>>>>>>>> this
>>>> change
>>>>>>>>>>>>>>>> either
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> week or early next week.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Let me know what you all
think.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> - Ian
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>

Mime
View raw message