hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lei Chang <lei_ch...@apache.org>
Subject Re: Replace git submodule with git clone + file with commit number?
Date Thu, 07 Jul 2016 09:31:25 GMT
Looks not everyone is on the same page for this thread.

I think all think binary dependencies are good. For most dependencies, we
use binary dependencies.

But for some dependencies, for example, gporca, plr et al, there are no
binary releases for these dependencies available. and fortunately, these
dependencies are all optional, and are only enabled by "configure" options.
It will not impact normal hawq build experience.

And if users want to build these optional dependencies by themselves, it is
too difficult. So I think what paul is doing is to automate the build
process for these special dependencies. From this aspect, this change is
really helpful.

Since this is blocking the first release, I think if there are no better
options, I'd like propose we move on and close this issue.

thoughts?

Thanks
Lei


On Thu, Jul 7, 2016 at 5:07 PM, Guo Gang <paulguo@gmail.com> wrote:

> At the earliest stage, people who want to build a hawq release with typical
> options need to manually do a lot of things, e.g.
> 1) Manually build libyarn and libhdfs
> 2) Manually download gporca, gpos, gp-xerces from github and select
>     proper versions,  and build them, then pass related path using
> environment variable
>     to hawq.
> 3) Find proper versions of pgcrypto and plr, and build them.
>
> All of the above is not easy for a developer who have no knowledge of this
> before.
> That was the motive for us to automate all of them.
>
>
> 2016-07-07 4:53 GMT+08:00 Ting(Goden) Yao <tyao@pivotal.io>:
>
> > I might have missed this - what's the rationale behind changing binary
> > dependency to git submodule (or git clone in this proposal) (e.g. gporca)
> > in the first place?
> >
> > On Wed, Jul 6, 2016 at 9:03 AM Roman Shaposhnik <roman@shaposhnik.org>
> > wrote:
> >
> > > On Wed, Jul 6, 2016 at 4:35 AM, Gmail <xunzhangthu@gmail.com> wrote:
> > > > I think the method using git clone is fine.
> > > >
> > > > But I suggest we'd better keep the makefile readable. I mean we
> should
> > > not add too trivial shell logic inside makefile itself.
> > > > For example, write a shell or Python script for users. Users should
> run
> > > the script before building HAWQ if they need to install this libraries.
> > >
> > > Huge +1 to the above. That's what I was suggesting in my previous
> > > emails saying that
> > > you should allow users to make dependencies available and then just
> > > point at them.
> > >
> > > > I also wonder what's the typical solution for similar problems of
> other
> > > Apache projects.
> > >
> > > Typically solution is to make sure that your build logic is capable of
> > > picking up
> > > binary dependencies.
> > >
> > > > Do they use git sub module?
> > >
> > > Not really. Most ASF projects have a single repo for the project (docs
> > > and site source
> > > code being an exception) and their dependency management is done
> through
> > > binary
> > > dependencies.
> > >
> > > > How do they solve this in their release?
> > >
> > > When you say 'release' this gets me slightly worried in the context of
> > > Git submodules.
> > > See, releases of ASF projects must be self-contained to a degree. If
> > > you require source
> > > in the Git submodule for the release to be functional it should give
> > > you a pretty strong
> > > indication that the source probably belongs to your project. If what
> > > submodules does
> > > is more of a plugin -- just declare a binary dependency.
> > >
> > > Thanks,
> > > Roman.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message