Mailing-List: contact dev-help@hawq.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hawq.incubator.apache.org
MIME-Version: 1.0
References: <CA+ULb+u+7k7fbGz1bmNAyATUfUrY-Lj8-BoSKD4RJAFOoZYzaw@mail.gmail.com>
 <CCDF5D3C-EB56-4CAB-A537-3509A65203BD@apache.org> <CA+ULb+stFoqh2-0TFNiyr9t8LmdHKNN5ZVfaAciMCr_E83eDGQ@mail.gmail.com>
 <6EAAE263-9555-4520-80F6-5A2112A5AFD4@apache.org> <CAOPnHqcA3p8KhcOcFZTbwTiG_+OxJUaL+YNZ20g=tSvMx9Yj0w@mail.gmail.com>
 <CAKgUS2BVFCXBN5-ccT1=N=y58XRuQiTUD6z_Mx-M6uwrffxTjw@mail.gmail.com> <F734237186262828.804D40A2-DAA4-47C4-8F87-773C2C85C8DF@mail.outlook.com>
In-Reply-To: <F734237186262828.804D40A2-DAA4-47C4-8F87-773C2C85C8DF@mail.outlook.com>
From: Kyle Dunn <kdunn@pivotal.io>
Date: Thu, 15 Sep 2016 04:54:27 +0000
Message-ID: <CADvk=Ga7=PB6fdVREUKH9+785pvRHHGQpVbbZNvKoJxhFv3gjg@mail.gmail.com>
Subject: Re: libhdfs3 development is still going on outside of ASF
To: dev@hawq.incubator.apache.org
Content-Type: multipart/alternative; boundary=94eb2c06f45c93f002053c84a4da
archived-at: Thu, 15 Sep 2016 04:54:51 -0000

--94eb2c06f45c93f002053c84a4da
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Chiming in here only as a casual but concerned observer.

Open source is about community first. If the logistics around "where"
libhdfs3 lives rather than the much more important issue of "how" it lives
are the focus here, I think we've missed the real issue.

For what it's worth, I concur with others, let's move it to HAWQ
exclusively and move on to addressing the community, starting with the
decision being made and how/where future contributions can be made.

My brief scan of libhdfs3 shows numerous open pull requests (with
apparently useful contributions) and several loose ends "issues". We need
to communicate effectively to these contributors whether those PRs and
issues are valuable and relevant. This type of engagement is what OSS
projects live and die by. We need to be better, starting with libhdfs3,
into HAWQ, and beyond.

"Open source isn't someone else's job" - it's everyone's job. I'm
challenging everyone with commit responsibly on repos to value community
input (both code and issues) as highly as your own backlog. Pay it forward
and maybe the community will start shrinking your backlog unexpectedly.


-Kyle

On Wed, Sep 14, 2016, 21:33 Lei Chang <chang.lei.cn@gmail.com> wrote:

>
> There was a short discussion before when we moved libhfds3 to HAWQ repo.
>
> http://mail-archives.apache.org/mod_mbox/incubator-hawq-dev/201602.mbox/%=
3cCAE44UQe1xgcVOC76T_mgVbgGbR=3DLx=3DXUBPVw18ZK4iZ3euCH+g@mail.gmail.com%3e
> I think it makes sense to keep libhdfs3 only in HAWQ repo to simplify
> Apache build and releases in current phase. This is what we have done in
> the past. But looks not everyone is on the same page.
> CheersLei
>
>
>
>
>
>
> On Thu, Sep 15, 2016 at 11:12 AM +0800, "Greg Chase" <greg@gregchase.com>
> wrote:
>
>
>
>
>
>
>
>
>
>
> Its fine if libhdfs3 is a third party license, and is treated that way.
>
> However, why does Apache HAWQ want to be dependent on some strange 3rd
> party library with no transparency?
>
> We are having enough difficulties just getting our first release out.
>
> Is there a compelling reason why we need to keep up with the independentl=
y
> developed libhdfs3 project?  Are they willing to make necessary changes s=
o
> that they are compatible with ASF's strict-for-a-good-reason policies?
>
> Can we fork hdfs3 for Apache HAWQ's purposes in Apache?
>
> If any libhdfs3 committers are also part of Apache HAWQ, perhaps you can
> shed some light on the viability of this as an independent project since =
I
> only see 4 contributors.
>
> -Greg
>
> On Wed, Sep 14, 2016 at 7:54 PM, Hong Wu  wrote:
>
> > In my opinion, I think it is reasonable to transfer the third-party rep=
o
> of
> > libhdfs3 totally into HAWQ, not only for the convenience of HAWQ build,
> but
> > also for the consideration of ASF project. So for HAWQ project, I am wi=
th
> > Roman.
> >
> > But my concern is the current users of libhdfs3 and all the pull
> requests,
> > wiki docs and issues. Another uncertain aspect from my perspective is
> that
> > although HAWQ could not run without libhdfs3, libhdfs3 could be used in
> > other open source projects, that might be the true meaning of making
> > libhdfs3 open source at the beginning.
> >
> > In summary, if it is really against the spirit of a ASF project for
> HAWQ, a
> > suggested way might be marking original libhdfs3 repo as a legacy repo =
in
> > stead of remove it.
> >
> > Best
> > Hong
> >
> > 2016-09-15 10:04 GMT+08:00 Zhanwei Wang :
> >
> > > Currently libhdfs3=E2=80=99s official code is not the same as in HAWQ=
. Some new
> > > code does not copy into HAWQ.  I do not think code change of libhdfs3
> > > should follow HAWQ=E2=80=99s commit process because  many change are =
not
> related
> > to
> > > HAWQ.
> > >
> > > From HAWQ side, I suggest to keep the stable version of its third-par=
ty
> > > libraries and copy new libhdfs3=E2=80=99s code only when it is necess=
ary.
> > >
> > > libhdfs3 was open source years before HAWQ incubating with a separate=
d
> > > permission of its authority. So in my opinion it is a third party and
> it
> > > actually was a third party before HAWQ incubating. And HAWQ is not th=
e
> > only
> > > user.
> > >
> > >
> > >
> > > Best Regards
> > >
> > > Zhanwei Wang
> > > wangzw@apache.org
> > >
> > >
> > >
> > > > =E5=9C=A8 2016=E5=B9=B49=E6=9C=8815=E6=97=A5=EF=BC=8C=E4=B8=8A=E5=
=8D=889:35=EF=BC=8CRoman Shaposhnik  =E5=86=99=E9=81=93=EF=BC=9A
> > > >
> > > > On Wed, Sep 14, 2016 at 6:29 PM, Zhanwei Wang
> > wrote:
> > > >> Hi Roman
> > > >>
> > > >> libhdfs3 works as third-party library of HAWQ, Just for the
> > convenience
> > > of HAWQ release
> > > >> process we copy its code into HAWQ.  The reason is that HAWQ used =
to
> > > dependent on
> > > >> specific version of libhdfs3 and libhdfs3 only distribute as sourc=
e
> > > code and the build process is complicated.
> > > >
> > > > I actually don't buy this argument. libhdfs3 is not an optional
> > > > dependency for HAWQ
> > > > like ORCA is (for example). Without libhdfs3 there's pretty tough t=
o
> > > > imagine HAWQ.
> > > > As such the code base needs to be governed as part of the ASF
> project,
> > > > not a random
> > > > GitHub dependency.
> > > >
> > > > IOW, let me ask you this: were all the changes that went into
> libhdfs3
> > > > that is part of
> > > > HAWQ discussed and reviewed via the ASF development process or did
> you
> > > just
> > > > import them from time to time as this comment suggests:
> > > >    https://issues.apache.org/jira/browse/HAWQ-1046?
> > > focusedCommentId=3D15489669&page=3Dcom.atlassian.jira.
> > > plugin.system.issuetabpanels:comment-tabpanel#comment-15489669
> > > > ?
> > > >
> > > >> I do not think we have any reason to shutdown a third party=E2=80=
=99s
> official
> > > repository.
> > > >
> > > > You say 3d party as though its not just you guys maintaining it on
> the
> > > side.
> > > >
> > > >> We also copy google test source code into HAWQ, just as what we di=
d
> > for
> > > libhdfs3.
> > > >
> > > > But this is very different. You don't do any development (certainly
> > > > you don't do any
> > > > non-trivial development) of that code.
> > > >
> > > >> libhdfs3 open source under Apache license version 2 just the same =
as
> > > HAWQ. So I believe there is no license issue.
> > > >
> > > > You're correct. There's no licensing issue but there's a pretty
> > > significant
> > > > governance issue.
> > > >
> > > > Thanks,
> > > > Roman.
> > > >
> > >
> > >
> >
>
>
>
>
>
> --
*Kyle Dunn | Data Engineering | Pivotal*
Direct: 303.905.3171 <3039053171> | Email: kdunn@pivotal.io

--94eb2c06f45c93f002053c84a4da--