Mailing-List: contact dev-help@asterixdb.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@asterixdb.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAN_YF5w797YvgSaeCLE0MyGKeKu54yZ1QXZ6qCUN+p31k3rXuw@mail.gmail.com>
References: 
 <CAN_YF5w0=Krfj0-frVzrFT-SivDnWrfQf6d8U0DKKp-E5p9=cA@mail.gmail.com>
 <71F2D731-2B41-49FC-9F27-1A00BB58C87F@apache.org>
 <CAN_YF5yuwjZAL1uDAeYapLK2_vp942Xw1suy_98zK5ZVPhijbQ@mail.gmail.com>
 <CAN_YF5yYN-20X3z5-y_M+i7+rotR07i8aY3Z0HeB6_2sgxr7Zw@mail.gmail.com>
 <F9FE3AB6-6B84-44E0-A4F0-D3C49126B49B@apache.org>
 <CAN_YF5w9wSDjaaFqH7OuLDO_YJ+XQbSJqj1VQ-360LouHJwMVg@mail.gmail.com>
 <84DF74BD-8BE2-42C0-B449-D9A18A29DC78@apache.org>
 <CAE1xAntJ0myHC0jG-Y4ouayuEw=3L8Co3LPZWNtrXXKa0YyShg@mail.gmail.com>
 <CAE1xAnvA0=uRMo8f9OEUkcQx-7+pQHLqzY-hdJuqUTgNNHzdqg@mail.gmail.com>
 <CAE1xAnsDYX+b_5ZSgQm2Laq6_CrndtPCFAPZv9viYZE7W06etQ@mail.gmail.com>
 <5702CBDE.5090407@gmail.com>
 <CAE1xAnugE-5K=XUptRkwuHrFjgNt+xOW4YqnoZ=m2v+uk9hvNQ@mail.gmail.com>
 <CAE1xAnuGbVPtxDgJXsRngxqgtU7jxM-LmcRg9q3KG+Y-5bomqg@mail.gmail.com>
 <CAN_YF5yT54XT52DWcnqB43Q1_m8DJdtZQ1bttRG4o14+dHHTWw@mail.gmail.com>
 <CAN_YF5zkcn3b6Wn3OPkJOe7GocsTr-eDHVL=e4hRO1uaumhoEg@mail.gmail.com>
 <CAN_YF5xrnGxzDLh-A7cDxOaokzksgO439cXh4sYm+KM6+BNq9g@mail.gmail.com>
 <CABytAGq3bwQ0c-8p5BaYXcQ5m+T6Mk1_djuz_H24a_ryVJEXKg@mail.gmail.com>
 <CAN_YF5w797YvgSaeCLE0MyGKeKu54yZ1QXZ6qCUN+p31k3rXuw@mail.gmail.com>
From: Taewoo Kim <wangsaeu@gmail.com>
Date: Thu, 7 Apr 2016 20:01:18 -0700
Message-ID: 
 <CABytAGoimFpR2hMbBGybGn8uteNrgy+QDnymaFXwKPAQnKQdbA@mail.gmail.com>
Subject: Re: The Great Merge
To: dev@asterixdb.incubator.apache.org
Content-Type: multipart/alternative; boundary=001a1140887edc3c4f052ff069ec

--001a1140887edc3c4f052ff069ec
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Thanks. I will try.

Best,
Taewoo

On Thu, Apr 7, 2016 at 7:58 PM, Ian Maxon <imaxon@uci.edu> wrote:

> Say if you had an AsterixDB change in a topic branch that had both
> AsterixDB and Hyracks components, it'd be like:
>
> git checkout (topic branch)
> git fetch origin
> git merge origin/master
> (resolve conflicts)
> git commit
> git remote add hyracks-local file://(absolute path to your hyracks repo)
> git fetch hyracks-local
> git merge hyracks-local/(name of your hyracks topic branch)
> (resolve conflicts)
>
>
> On Thu, Apr 7, 2016 at 7:31 PM, Taewoo Kim <wangsaeu@gmail.com> wrote:
>
> > @Ian:
> >
> > Could you show the process (the explicit command, if possible) step by
> > step? I know you did. But, I have a hard time to apply your merge. Than=
k
> > you. Steven's patch should work, also. But it didn't work on my branch.
> It
> > looks like I miss something.
> >
> >
> >
> > Best,
> > Taewoo
> >
> > On Thu, Apr 7, 2016 at 9:12 AM, Ian Maxon <imaxon@uci.edu> wrote:
> >
> > > Also, please don't push any new changes to Hyracks. You will need to
> > merge
> > > these into your asterix change or a new change if it is a hyracks
> change
> > > with no asterixdb content. I have disabled the jenkins job that
> verifies
> > > hyracks patches on Gerrit so hopefully nothing gets through.
> > >
> > > On Thu, Apr 7, 2016 at 8:02 AM, Ian Maxon <imaxon@uci.edu> wrote:
> > >
> > > > Alright, everything is pushed. Please post here or start another
> thread
> > > if
> > > > you start to experience issues.
> > > >
> > > > On Tue, Apr 5, 2016 at 12:21 PM, Ian Maxon <imaxon@uci.edu> wrote:
> > > >
> > > >> Just as a reminder, I'll be merging this tommorow since it seems
> like
> > > >> everything should go fairly smoothly.
> > > >>
> > > >> On Mon, Apr 4, 2016 at 1:20 PM, Steven Jacobs <sjaco002@ucr.edu>
> > wrote:
> > > >>
> > > >>> http://landbeforetime.wikia.com/wiki/Great_Valley
> > > >>>
> > > >>> On Mon, Apr 4, 2016 at 1:19 PM, Steven Jacobs <sjaco002@ucr.edu>
> > > wrote:
> > > >>>
> > > >>> > No, we are living in the GREAT valley :)
> > > >>> > Steven
> > > >>> >
> > > >>> > On Mon, Apr 4, 2016 at 1:17 PM, Mike Carey <dtabass@gmail.com>
> > > wrote:
> > > >>> >
> > > >>> >> Sounds like things are GOOD!  Excellent.  (So not to be feared
> > like
> > > >>> the
> > > >>> >> event that the name of this one keeps reminding me of:
> > > >>> >> http://landbeforetime.wikia.com/wiki/Great_Earthshake :-).)
> > > >>> >>
> > > >>> >>
> > > >>> >> On 4/4/16 1:12 PM, Steven Jacobs wrote:
> > > >>> >>
> > > >>> >>> It seems that I might be the only one concerned here, but it
> > seems
> > > >>> like
> > > >>> >>> there should be others, so I am continuing this thread.
> > > >>> >>>
> > > >>> >>> I modified the perl REGEX from Chris' summer solution, and it
> > > works!
> > > >>> >>>
> > > >>> >>> Once Ian has merged master:
> > > >>> >>>
> > > >>> >>> 1. On your local branch, find the *parent* of the first commi=
t
> > you
> > > >>> want
> > > >>> >>> to
> > > >>> >>> migrate onto the new master, e.g.
> > > >>> >>> de6e0da24c26037967eb9a937d2c77c6c43e8761
> > > >>> >>>
> > > >>> >>> 2. Run this magic command:
> > > >>> >>>
> > > >>> >>>     git format-patch --stdout
> > > >>> de6e0da24c26037967eb9a937d2c77c6c43e8761 |
> > > >>> >>> perl -pe 's#asterix-#asterixdb/asterix-#g' > /tmp/my.patch
> > > >>> >>>
> > > >>> >>> 3. Now fetch master, and create a new local branch from it:
> > > >>> >>>
> > > >>> >>>     git switch master; git pull; git checkout -B newbranch
> > > >>> >>>
> > > >>> >>> 4. Apply your tweaked patch:
> > > >>> >>>
> > > >>> >>>     git am /tmp/my.patch
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> This recognized ALL of my file moves/renames and applied them
> > > >>> correctly.
> > > >>> >>> It
> > > >>> >>> leaves only two issues:
> > > >>> >>> 1) Something similar will probably need to be done for Hyrack=
s
> > > >>> changes
> > > >>> >>> 2) My pom changes didn't apply. This isn't so bad since there
> are
> > > >>> only a
> > > >>> >>> few pom files total.
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> I hope this helps,
> > > >>> >>> Steven
> > > >>> >>>
> > > >>> >>>
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> On Fri, Apr 1, 2016 at 11:31 AM, Steven Jacobs <
> sjaco002@ucr.edu
> > >
> > > >>> wrote:
> > > >>> >>>
> > > >>> >>> Here is Chris's original solution to give context. I think
> > changing
> > > >>> the
> > > >>> >>>> REGEX might be enough to re-use the solution:
> > > >>> >>>>
> > > >>> >>>> 1. On your local branch, find the *parent* of the first comm=
it
> > you
> > > >>> want
> > > >>> >>>> to
> > > >>> >>>> migrate onto the new master. If you were fully up-to-date
> before
> > > the
> > > >>> >>>> repackaging commits went in, this will be Till's
> > > >>> >>>> change 95350e253f3462b1fb8d08396b4fddadaa33bf53, so I'll use
> > that
> > > >>> here.
> > > >>> >>>>
> > > >>> >>>> 2. Run this magic command:
> > > >>> >>>>
> > > >>> >>>>     git format-patch --stdout
> > > >>> 95350e253f3462b1fb8d08396b4fddadaa33bf53 |
> > > >>> >>>> perl -pe 's#edu(.)uci.ics#org\1apache#g' > /tmp/my.patch
> > > >>> >>>>
> > > >>> >>>> 3. Now fetch the new master, and create a new local branch
> from
> > > it:
> > > >>> >>>>
> > > >>> >>>>     git switch master; git pull; git checkout -B newbranch
> > > >>> >>>>
> > > >>> >>>> 4. Apply your tweaked patch:
> > > >>> >>>>
> > > >>> >>>>     git am /tmp/my.patch
> > > >>> >>>>
> > > >>> >>>>
> > > >>> >>>> Steven
> > > >>> >>>>
> > > >>> >>>> On Fri, Apr 1, 2016 at 11:07 AM, Steven Jacobs <
> > sjaco002@ucr.edu>
> > > >>> >>>> wrote:
> > > >>> >>>>
> > > >>> >>>> I've tried doing this now on my branch.
> > > >>> >>>>> As I feared, all of the files that are renamed/moved become
> > > >>> conflicts
> > > >>> >>>>> (just a few hundred conflicts in my case =F0=9F=98=91).
> > > >>> >>>>> I'm wondering if we could use a similar technique for what =
we
> > did
> > > >>> >>>>> during
> > > >>> >>>>> the summer (for the apache change) to get around this.
> > > >>> >>>>>
> > > >>> >>>>> Steven
> > > >>> >>>>>
> > > >>> >>>>> On Fri, Apr 1, 2016 at 9:40 AM, Till Westmann <
> > tillw@apache.org>
> > > >>> >>>>> wrote:
> > > >>> >>>>>
> > > >>> >>>>> I=E2=80=99m not sure I completely understand what you are s=
aying. Is
> > > this a
> > > >>> >>>>>> temporary state that will get cleaned up later or is this
> > > >>> supposed to
> > > >>> >>>>>> stay this way (having "-fullstack" in the names)?
> > > >>> >>>>>>
> > > >>> >>>>>> Thanks,
> > > >>> >>>>>> Till
> > > >>> >>>>>>
> > > >>> >>>>>>
> > > >>> >>>>>> On 31 Mar 2016, at 19:39, Ian Maxon wrote:
> > > >>> >>>>>>
> > > >>> >>>>>> I'm not sure if it was necessary to rename it, but the
> > original
> > > >>> issue
> > > >>> >>>>>> is
> > > >>> >>>>>>
> > > >>> >>>>>>> that the hyracks repo itself has a folder named hyracks,
> that
> > > >>> >>>>>>> contains
> > > >>> >>>>>>> hyracks. I thought this might confuse git if I did
> something
> > > like
> > > >>> >>>>>>> make a
> > > >>> >>>>>>> new temporary folder, move everything into that, and then
> > > rename
> > > >>> it
> > > >>> >>>>>>> to
> > > >>> >>>>>>> 'hyracks'.
> > > >>> >>>>>>>
> > > >>> >>>>>>> On Thu, Mar 31, 2016 at 6:35 PM, Till Westmann <
> > > tillw@apache.org
> > > >>> >
> > > >>> >>>>>>> wrote:
> > > >>> >>>>>>>
> > > >>> >>>>>>> Interesting!
> > > >>> >>>>>>>
> > > >>> >>>>>>>> One thing I=E2=80=99m wondering about is why you=E2=80=
=99ve added
> > "-fullstack"
> > > >>> to
> > > >>> >>>>>>>> the
> > > >>> >>>>>>>> artifactId and the hyracks module.
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> Cheers,
> > > >>> >>>>>>>> Till
> > > >>> >>>>>>>>
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> On 31 Mar 2016, at 17:21, Ian Maxon wrote:
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> I've gone ahead and tried merging my topic branch with
> this
> > > >>> change,
> > > >>> >>>>>>>> and it
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> turned out surprisingly well. I really didn't have many
> > > issues.
> > > >>> I'll
> > > >>> >>>>>>>>> summarize the process:
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> 1) Merge the change from asterixdb with your topic bran=
ch
> > > >>> checked
> > > >>> >>>>>>>>> out, so
> > > >>> >>>>>>>>> just 'git merge hyracks-merge2'.
> > > >>> >>>>>>>>> The only real conflict should be the pom, if you altere=
d
> > > that.
> > > >>> I
> > > >>> >>>>>>>>> found it
> > > >>> >>>>>>>>> easiest to just replicate my changes and take the
> upstream,
> > > >>> rather
> > > >>> >>>>>>>>> than
> > > >>> >>>>>>>>> trying anything funny, since usually pom changes are no=
t
> > > major.
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> 2) Add your hyracks folder as a remote (for me, 'git
> remote
> > > add
> > > >>> >>>>>>>>> hyracks-local file:///home/...')
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> 3) Merge your hyracks topic branch into asterixdb ( ' g=
it
> > > merge
> > > >>> >>>>>>>>> hyracks-local/imaxon/hdfs')
> > > >>> >>>>>>>>> This also worked pretty well, the only extra hiccup
> besides
> > > >>> the pom
> > > >>> >>>>>>>>> was
> > > >>> >>>>>>>>> files I had created. Those appeared at the top level
> again
> > > >>> after
> > > >>> >>>>>>>>> the
> > > >>> >>>>>>>>> merge.
> > > >>> >>>>>>>>> But, all you have to do is move them back down one fold=
er
> > > into
> > > >>> >>>>>>>>> hyracks-fullstack.
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> That's about it really. I went ahead and pushed this up
> to
> > > >>> github
> > > >>> >>>>>>>>> as
> > > >>> >>>>>>>>> well
> > > >>> >>>>>>>>> so if anyone would like to take a look at the process o=
r
> > > check
> > > >>> out
> > > >>> >>>>>>>>> the
> > > >>> >>>>>>>>> branch to see what happened (at least for me), the bran=
ch
> > is
> > > >>> here:
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>>
> > > >>>
> > >
> >
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/hdfs-plus-h=
yracks
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> Thanks,
> > > >>> >>>>>>>>> -Ian
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> On Wed, Mar 30, 2016 at 6:17 PM, Ian Maxon <
> imaxon@uci.edu
> > >
> > > >>> wrote:
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> Chris found an issue with the way git histories were
> being
> > > >>> handled
> > > >>> >>>>>>>>> in
> > > >>> >>>>>>>>> the
> > > >>> >>>>>>>>>
> > > >>> >>>>>>>>> way I merged things, so I have revised the proposed
> branch:
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>>
> > >
> https://github.com/parshimers/incubator-asterixdb/commits/hyracks-merge2
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> Basically I was trying to fit everything into one
> commit,
> > > >>> because
> > > >>> >>>>>>>>>> I
> > > >>> >>>>>>>>>> thought at first that I could submit it to Gerrit that
> > way.
> > > >>> >>>>>>>>>> However
> > > >>> >>>>>>>>>> that
> > > >>> >>>>>>>>>> doesn't work for other reasons, basically Gerrit tries
> to
> > > >>> treat
> > > >>> >>>>>>>>>> every new
> > > >>> >>>>>>>>>> commit from Hyracks as a new change. Splitting the
> commits
> > > of
> > > >>> the
> > > >>> >>>>>>>>>> repository merge fixes the issue.
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> @Till, I think that creating a textual patch would jus=
t
> be
> > > >>> more
> > > >>> >>>>>>>>>> work. If
> > > >>> >>>>>>>>>> I
> > > >>> >>>>>>>>>> were to do it that way I would try fetching the Gerrit
> > > patch,
> > > >>> and
> > > >>> >>>>>>>>>> then
> > > >>> >>>>>>>>>> cherry-picking it onto a new branch that has the
> > > >>> hyracks+asterix
> > > >>> >>>>>>>>>> master
> > > >>> >>>>>>>>>> as
> > > >>> >>>>>>>>>> the head.
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> On Wed, Mar 30, 2016 at 5:42 PM, Till Westmann <
> > > >>> tillw@apache.org>
> > > >>> >>>>>>>>>> wrote:
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> To get existing patches in, could we just create a
> textual
> > > >>> patch
> > > >>> >>>>>>>>>> (e.g.
> > > >>> >>>>>>>>>>
> > > >>> >>>>>>>>>> from gerrit), apply that with the necessary -p option
> to a
> > > new
> > > >>> >>>>>>>>>>> local
> > > >>> >>>>>>>>>>> checkout of the merged repositories and submit a new
> > review
> > > >>> to
> > > >>> >>>>>>>>>>> gerrit?
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> Thanks,
> > > >>> >>>>>>>>>>> Till
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> On 30 Mar 2016, at 12:36, Ian Maxon wrote:
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> Hi all,
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> I went ahead preliminarily merged the Hyracks and
> > AsterixDB
> > > >>> >>>>>>>>>>>> repositories
> > > >>> >>>>>>>>>>>> into one. Unfortunately this can't be reviewed in
> Gerrit
> > > so
> > > >>> you
> > > >>> >>>>>>>>>>>> all can
> > > >>> >>>>>>>>>>>> check it out here:
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>
> > > >>>
> > >
> >
> https://github.com/parshimers/incubator-asterixdb/tree/imaxon/merge-hyrac=
ks
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>> You will likely have to do some ugly rebasing for
> > whatever
> > > >>> >>>>>>>>>>>> changes
> > > >>> >>>>>>>>>>>> you
> > > >>> >>>>>>>>>>>> might have open once this gets done, since it moves
> > > >>> asterixdb
> > > >>> >>>>>>>>>>>> down
> > > >>> >>>>>>>>>>>> one
> > > >>> >>>>>>>>>>>> folder and swaps out pom.xml in the repository root.
> > > >>> Hyracks is
> > > >>> >>>>>>>>>>>> in
> > > >>> >>>>>>>>>>>> a
> > > >>> >>>>>>>>>>>> similar situation, though you would want to reapply
> your
> > > >>> change
> > > >>> >>>>>>>>>>>> to
> > > >>> >>>>>>>>>>>> the
> > > >>> >>>>>>>>>>>> AsterixDB repo from Hyracks (which is a bit odd). If
> you
> > > >>> would
> > > >>> >>>>>>>>>>>> like to
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> see
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>> how this affects your branch please do try fetching t=
he
> > > >>> branch I
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>>> linked
> > > >>> >>>>>>>>>>>> above and testing it out on a copy of your topic
> branch.
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> I'm still making sure all of the tests pass but
> > nothing's
> > > >>> >>>>>>>>>>>> failed so
> > > >>> >>>>>>>>>>>> far.
> > > >>> >>>>>>>>>>>> Unless anyone has objections I think we should push
> this
> > > >>> change
> > > >>> >>>>>>>>>>>> either
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> this
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>> week or early next week.
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>>> Let me know what you all think.
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>> Thanks,
> > > >>> >>>>>>>>>>>> - Ian
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>>
> > > >>> >>>>>>>>>>>
> > > >>> >>>>>>>>>>
> > > >>> >>
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >
>

--001a1140887edc3c4f052ff069ec--