asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Carey <dtab...@gmail.com>
Subject Re: [WARNING] Don't push anything into the hyracks git repository
Date Thu, 07 Apr 2016 18:28:51 GMT
PHEW!  Stopped me just in time.  :-)

On 4/7/16 10:24 AM, Till Westmann wrote:
> Jut a forward for people (like myself) that don’t always see which 
> messages are important :)
>
> Cheers,
> Till
>
> Forwarded message:
>
>> From: Ian Maxon <imaxon@uci.edu>
>> To: dev@asterixdb.incubator.apache.org
>> Subject: Re: The Great Merge
>> Date: Thu, 7 Apr 2016 09:12:40 -0700
>>
>> Also, please don't push any new changes to Hyracks. You will need to 
>> merge
>> these into your asterix change or a new change if it is a hyracks change
>> with no asterixdb content. I have disabled the jenkins job that verifies
>> hyracks patches on Gerrit so hopefully nothing gets through.
>>
>> On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <imaxon@uci.edu> wrote:
>>
>>> Alright, everything is pushed. Please post here or start another 
>>> thread if
>>> you start to experience issues.
>>>
>>> On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <imaxon@uci.edu> wrote:
>>>
>>>> Just as a reminder, I'll be merging this tommorow since it seems like
>>>> everything should go fairly smoothly.
>>>>
>>>> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sjaco002@ucr.edu> 
>>>> wrote:
>>>>
>>>>> http://landbeforetime.wikia.com/wiki/Great_Valley
>>>>>
>>>>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sjaco002@ucr.edu>

>>>>> wrote:
>>>>>
>>>>>> No, we are living in the GREAT valley :)
>>>>>> Steven
>>>>>>
>>>>>> On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dtabass@gmail.com>

>>>>>> wrote:
>>>>>>
>>>>>>> Sounds like things are GOOD! Excellent.  (So not to be feared
like
>>>>> the
>>>>>>> event that the name of this one keeps reminding me of:
>>>>>>> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
>>>>>>>
>>>>>>>
>>>>>>> On 4/4/16 1:12 PM, Steven Jacobs wrote:
>>>>>>>
>>>>>>>> It seems that I might be the only one concerned here, but
it seems
>>>>> like
>>>>>>>> there should be others, so I am continuing this thread.
>>>>>>>>
>>>>>>>> I modified the perl REGEX from Chris' summer solution, and
it 
>>>>>>>> works!
>>>>>>>>
>>>>>>>> Once Ian has merged master:
>>>>>>>>
>>>>>>>> 1. On your local branch, find the *parent* of the first commit
you
>>>>> want
>>>>>>>> to
>>>>>>>> migrate onto the new master, e.g.
>>>>>>>> de6e0da24c26037967eb9a937d2c77c6c43e8761
>>>>>>>>
>>>>>>>> 2. Run this magic command:
>>>>>>>>
>>>>>>>>     git format-patch --stdout
>>>>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
>>>>>>>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
>>>>>>>>
>>>>>>>> 3. Now fetch master, and create a new local branch from it:
>>>>>>>>
>>>>>>>>     git switch master; git pull; git checkout -B newbranch
>>>>>>>>
>>>>>>>> 4. Apply your tweaked patch:
>>>>>>>>
>>>>>>>>     git am /tmp/my.patch
>>>>>>>>
>>>>>>>>
>>>>>>>> This recognized ALL of my file moves/renames and applied
them
>>>>> correctly.
>>>>>>>> It
>>>>>>>> leaves only two issues:
>>>>>>>> 1) Something similar will probably need to be done for Hyracks
>>>>> changes
>>>>>>>> 2) My pom changes didn't apply. This isn't so bad since there
are
>>>>> only a
>>>>>>>> few pom files total.
>>>>>>>>
>>>>>>>>
>>>>>>>> I hope this helps,
>>>>>>>> Steven
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <sjaco002@ucr.edu>
>>>>> wrote:
>>>>>>>>
>>>>>>>> Here is Chris's original solution to give context. I think

>>>>>>>> changing
>>>>> the
>>>>>>>>> REGEX might be enough to re-use the solution:
>>>>>>>>>
>>>>>>>>> 1. On your local branch, find the *parent* of the first
commit 
>>>>>>>>> you
>>>>> want
>>>>>>>>> to
>>>>>>>>> migrate onto the new master. If you were fully up-to-date

>>>>>>>>> before the
>>>>>>>>> repackaging commits went in, this will be Till's
>>>>>>>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll
use that
>>>>> here.
>>>>>>>>>
>>>>>>>>> 2. Run this magic command:
>>>>>>>>>
>>>>>>>>>     git format-patch --stdout
>>>>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
>>>>>>>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
>>>>>>>>>
>>>>>>>>> 3. Now fetch the new master, and create a new local branch

>>>>>>>>> from it:
>>>>>>>>>
>>>>>>>>>     git switch master; git pull; git checkout -B newbranch
>>>>>>>>>
>>>>>>>>> 4. Apply your tweaked patch:
>>>>>>>>>
>>>>>>>>>     git am /tmp/my.patch
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Steven
>>>>>>>>>
>>>>>>>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <sjaco002@ucr.edu>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I've tried doing this now on my branch.
>>>>>>>>>> As I feared, all of the files that are renamed/moved
become
>>>>> conflicts
>>>>>>>>>> (just a few hundred conflicts in my case 😑).
>>>>>>>>>> I'm wondering if we could use a similar technique
for what we 
>>>>>>>>>> did
>>>>>>>>>> during
>>>>>>>>>> the summer (for the apache change) to get around
this.
>>>>>>>>>>
>>>>>>>>>> Steven
>>>>>>>>>>
>>>>>>>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <tillw@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I’m not sure I completely understand what you are
saying. Is 
>>>>>>>>>> this a
>>>>>>>>>>> temporary state that will get cleaned up later
or is this
>>>>> supposed to
>>>>>>>>>>> stay this way (having "-fullstack" in the names)?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Till
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
>>>>>>>>>>>
>>>>>>>>>>> I'm not sure if it was necessary to rename it,
but the original
>>>>> issue
>>>>>>>>>>> is
>>>>>>>>>>>
>>>>>>>>>>>> that the hyracks repo itself has a folder
named hyracks, that
>>>>>>>>>>>> contains
>>>>>>>>>>>> hyracks. I thought this might confuse git
if I did 
>>>>>>>>>>>> something like
>>>>>>>>>>>> make a
>>>>>>>>>>>> new temporary folder, move everything into
that, and then 
>>>>>>>>>>>> rename
>>>>> it
>>>>>>>>>>>> to
>>>>>>>>>>>> 'hyracks'.
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann

>>>>>>>>>>>> <tillw@apache.org
>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Interesting!
>>>>>>>>>>>>
>>>>>>>>>>>>> One thing I’m wondering about is why
you’ve added 
>>>>>>>>>>>>> "-fullstack"
>>>>> to
>>>>>>>>>>>>> the
>>>>>>>>>>>>> artifactId and the hyracks module.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>> Till
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've gone ahead and tried merging my
topic branch with this
>>>>> change,
>>>>>>>>>>>>> and it
>>>>>>>>>>>>>
>>>>>>>>>>>>> turned out surprisingly well. I really
didn't have many 
>>>>>>>>>>>>> issues.
>>>>> I'll
>>>>>>>>>>>>>> summarize the process:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) Merge the change from asterixdb
with your topic branch
>>>>> checked
>>>>>>>>>>>>>> out, so
>>>>>>>>>>>>>> just 'git merge hyracks-merge2'.
>>>>>>>>>>>>>> The only real conflict should be
the pom, if you altered 
>>>>>>>>>>>>>> that.
>>>>> I
>>>>>>>>>>>>>> found it
>>>>>>>>>>>>>> easiest to just replicate my changes
and take the upstream,
>>>>> rather
>>>>>>>>>>>>>> than
>>>>>>>>>>>>>> trying anything funny, since usually
pom changes are not 
>>>>>>>>>>>>>> major.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2) Add your hyracks folder as a remote
(for me, 'git 
>>>>>>>>>>>>>> remote add
>>>>>>>>>>>>>> hyracks-local file:///home/...')
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 3) Merge your hyracks topic branch
into asterixdb ( ' git 
>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>> hyracks-local/imaxon/hdfs')
>>>>>>>>>>>>>> This also worked pretty well, the
only extra hiccup besides
>>>>> the pom
>>>>>>>>>>>>>> was
>>>>>>>>>>>>>> files I had created. Those appeared
at the top level again
>>>>> after
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> merge.
>>>>>>>>>>>>>> But, all you have to do is move them
back down one folder 
>>>>>>>>>>>>>> into
>>>>>>>>>>>>>> hyracks-fullstack.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That's about it really. I went ahead
and pushed this up to
>>>>> github
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>> well
>>>>>>>>>>>>>> so if anyone would like to take a
look at the process or 
>>>>>>>>>>>>>> check
>>>>> out
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> branch to see what happened (at least
for me), the branch is
>>>>> here:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-hyracks

>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -Ian
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM,
Ian Maxon <imaxon@uci.edu>
>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Chris found an issue with the way
git histories were being
>>>>> handled
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> way I merged things, so I have revised
the proposed branch:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2

>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Basically I was trying to fit
everything into one commit,
>>>>> because
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>> thought at first that I could
submit it to Gerrit that way.
>>>>>>>>>>>>>>> However
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> doesn't work for other reasons,
basically Gerrit tries to
>>>>> treat
>>>>>>>>>>>>>>> every new
>>>>>>>>>>>>>>> commit from Hyracks as a new
change. Splitting the 
>>>>>>>>>>>>>>> commits of
>>>>> the
>>>>>>>>>>>>>>> repository merge fixes the issue.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> @Till, I think that creating
a textual patch would just be
>>>>> more
>>>>>>>>>>>>>>> work. If
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>> were to do it that way I would
try fetching the Gerrit 
>>>>>>>>>>>>>>> patch,
>>>>> and
>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>> cherry-picking it onto a new
branch that has the
>>>>> hyracks+asterix
>>>>>>>>>>>>>>> master
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>> the head.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Mar 30, 2016 at 5:42
PM, Till Westmann <
>>>>> tillw@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To get existing patches in, could
we just create a textual
>>>>> patch
>>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> from gerrit), apply that with
the necessary -p option to 
>>>>>>>>>>>>>>> a new
>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>> checkout of the merged repositories
and submit a new 
>>>>>>>>>>>>>>>> review
>>>>> to
>>>>>>>>>>>>>>>> gerrit?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Till
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 30 Mar 2016, at 12:36,
Ian Maxon wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I went ahead preliminarily
merged the Hyracks and 
>>>>>>>>>>>>>>>> AsterixDB
>>>>>>>>>>>>>>>>> repositories
>>>>>>>>>>>>>>>>> into one. Unfortunately
this can't be reviewed in 
>>>>>>>>>>>>>>>>> Gerrit so
>>>>> you
>>>>>>>>>>>>>>>>> all can
>>>>>>>>>>>>>>>>> check it out here:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyracks

>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You will likely have to do
some ugly rebasing for whatever
>>>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> might have open once
this gets done, since it moves
>>>>> asterixdb
>>>>>>>>>>>>>>>>> down
>>>>>>>>>>>>>>>>> one
>>>>>>>>>>>>>>>>> folder and swaps out
pom.xml in the repository root.
>>>>> Hyracks is
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> similar situation, though
you would want to reapply your
>>>>> change
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> AsterixDB repo from Hyracks
(which is a bit odd). If you
>>>>> would
>>>>>>>>>>>>>>>>> like to
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> how this affects your branch
please do try fetching the
>>>>> branch I
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> linked
>>>>>>>>>>>>>>>>> above and testing it
out on a copy of your topic branch.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I'm still making sure
all of the tests pass but nothing's
>>>>>>>>>>>>>>>>> failed so
>>>>>>>>>>>>>>>>> far.
>>>>>>>>>>>>>>>>> Unless anyone has objections
I think we should push this
>>>>> change
>>>>>>>>>>>>>>>>> either
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> week or early next week.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Let me know what you
all think.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> - Ian
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message