hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guo Gang <paul...@gmail.com>
Subject Re: Replace git submodule with git clone + file with commit number?
Date Thu, 30 Jun 2016 10:51:45 GMT
Discussed more with Jiali, I realised that to make hawk repo be more
friendly to gporca developers, I could store git address, branch and a
commit number in one file so that gporca developers could easier switch
gporca code.

2016-06-30 15:35 GMT+08:00 Guo Gang <paulguo@gmail.com>:

> 1. HAWQ repo is not a good place for gp-xerces development. My change just
>    use a stable and working version of gp-xerces. If gp-xerces developers
> want to
>    test it with hawq, they can easily modify code under hawk: gp-xerces
> (e.g. pull from
>     a branch, etc.), then remove the timestamp file
> depends/thirdpaty/gp-xerces_build_timestamp
>    to trigger a rebuild.
>
> 2. Correct.
>
> 2016-06-30 11:31 GMT+08:00 Jiali Yao <jyao@pivotal.io>:
>
>> Hi Paul
>>
>> Generally agree on the approach while I have some questions:
>>
>> 1. Do we need to add branch handling?  For example, I am a developer
>> of gp-xerces,
>>  I need to test my branch code.
>>
>> 2. Let me double confirm about the build process.  Below is gp-xerces repo
>> and now HAWQ will be commit a. So I will set commit file gp-xerces.commit
>> in HAWQ repo which indicate we will use commit a. And then sometime later,
>> we want to use b, then we commit   gp-xerces.commit to b right?  And we
>> will have multiple .commit file for each submodule now. right?
>> gp-xerces:
>>      commit c
>>      ....
>>      commit b
>>      ....
>>      commit a
>>
>> Thanks
>> Jiali
>>
>> On Thu, Jun 30, 2016 at 11:03 AM, Radar Da lei <rlei@pivotal.io> wrote:
>>
>> > +1
>> > Since 'git clone' is called automatically in Makefile, I feel this is
>> much
>> > better than use submodule.
>> >
>> > Regards,
>> > Radar
>> >
>> > On Thu, Jun 30, 2016 at 10:07 AM, Lili Ma <lma@pivotal.io> wrote:
>> >
>> > > Hi Paul,
>> > >
>> > > I have one question, you mentioned "if I update gp-xerces commit
>> number
>> > in
>> > > the commit file gp-xerces.commit make will trigger an auto-build".
>> Could
>> > > we set the $(gp_xerces_commit) to any commit number? What I mean is,
>> > could
>> > > we set it to a commit number already existing in my local code, or
>> from
>> > > remote code which I have not fetched?
>> > >
>> > > If the answer is yes, I totally agree this suggestion.
>> > >
>> > > Thanks
>> > > Lili
>> > >
>> > > On Wed, Jun 29, 2016 at 11:52 PM, Kavinder Dhaliwal <
>> > kdhaliwal@pivotal.io>
>> > > wrote:
>> > >
>> > > > +1. I am in favor of this approach, especially if the submodules
>> make
>> > the
>> > > > source tarball difficult to build. I also agree it will make a
>> > developers
>> > > > life much easier.
>> > > >
>> > > > On Wed, Jun 29, 2016 at 3:18 AM, Guo Gang <paulguo@gmail.com>
>> wrote:
>> > > >
>> > > > > I'm proposing this change because:
>> > > > >
>> > > > > 1) We are ready to release the first apache release with "source
>> > > > tarball",
>> > > > > but
>> > > > >     submodule is not friendly to source tarball since git
>> submodule
>> > > > >    require a git parent.
>> > > > >
>> > > > > 2) With more and more development, I found that submodule
>> mechanism
>> > > > >     is not that friendly for development, e.g.
>> > > > >
>> > > > >     If the commit number of one submodule is modified, it is
hard
>> to
>> > > > detect
>> > > > >     this for Makefile. We need to manually update it in an old
>> repo.
>> > If
>> > > > >     using "git clone", we could easily detect update by set a
>> commit
>> > > > number
>> > > > > file
>> > > > >     as dependency and thus easily update the code.
>> > > > >
>> > > > >    Some developers have complained the annoying output in "git
>> > status"
>> > > > >    after building submodules. With "git clone" we can easily
mask
>> > those
>> > > > >    directories via .gitignore.
>> > > > >
>> > > > >    It is hard for developers who is not familiar with submodule
>> > > (Frankly
>> > > > > speaking
>> > > > >    I really do not think submodule is friendly) to manipulate
>> related
>> > > > > directories
>> > > > >    when necessary.
>> > > > >
>> > > > > While with the "git clone" solution, we save the commit of
>> previous
>> > > > > submodule in a file,
>> > > > > and the related Makefile code change is rather simple, e.g. for
>> > > > gp-xerces.
>> > > > >
>> > > > > $(ORCA_BLD_PATH)/gp-xerces_prepare_timestamp:
>> > > > > $(ORCA_SRC_PATH)/gp-xerces.commit
>> > > > >     rm -f $(ORCA_BLD_PATH)/gp-xerces_prepare_timestamp
>> > > > >
>> > > > > gp-xerces_prepare: $(ORCA_BLD_PATH)/gp-xerces_prepare_timestamp
>> > > > >     if [ ! -f $(ORCA_BLD_PATH)/gp-xerces_prepare_timestamp ];
>> then \
>> > > > >         [ "x$(gp_xerces_commit)" != "x" ] || exit 1; \
>> > > > >         cd $(abs_top_srcdir)/$(subdir); mkdir -p gp-xerces; cd
>> > > > gp-xerces; \
>> > > > >         [ ! -d .git ] && git clone
>> > > > > https://github.com/greenplum-db/gp-xerces.git .; \
>> > > > >         git reset --hard $(gp_xerces_commit) || exit 2; \
>> > > > >         touch $(ORCA_BLD_PATH)/gp_xerces_prepare_timestamp; \
>> > > > >     fi
>> > > > >
>> > > > > With above code change, if I update gp-xerces commit number in
the
>> > > commit
>> > > > > file
>> > > > > gp-xerces.commit make will trigger an auto-build, If I messed
up
>> the
>> > > > > gp-xerces directory
>> > > > > I can easily remove the whole gp-xerces, or just remove the
>> timestamp
>> > > > file
>> > > > > gp-xerces_prepare_timestamp to trigger an auto build.
>> > > > >
>> > > > > Any suggestion? Thanks.
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message