apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Re: A proposal for Malhar
Date Fri, 27 May 2016 16:52:51 GMT
Hi Hitesh,

Thanks for bringing out these points. Contrib is being published as part of
Malhar today with all the standards you mentioned below just like any other
module in Malhar and that will not change even if it will be repurposed for
the proposed use.

On Fri, May 27, 2016 at 9:45 AM, Hitesh Shah <hitesh@apache.org> wrote:

> Some general comments/questions:
>    - I am assuming that the contrib operators will be published as part of
> the release as well as their respective maven artifacts? This implies the
> same set of strict licensing/legal checks.
>    - Provenance of where the source came from as well as its license
> should be very clear. If there are donations of operators from an non-ALv2
> source, they might need to go through the SGA process.
>    - All dependencies of this operator also need to be of a compliant
> license.
>    - The community ( not just the contributor ) as a whole is taking
> responsibility for maintaining/supporting all of these contrib modules. It
> has been seen in the past with other projects that such contrib repos end
> up getting less eyeballs and are not generally maintained well especially
> if they are not part of the mainstream usage and end up as a dumping ground
> for all kinds of prototypes/proof-of-concepts.
>    - The contributions can be more permissive from a code quality point of
> view but not for the conditions listed above.
>
> thanks
> — Hitesh
>
>
> > On May 26, 2016, at 11:25 PM, Pramod Immaneni <pramod@datatorrent.com>
> wrote:
> >
> > As you all know the continued success and growth of an open source
> project
> > is dependent on new members joining the project and contributing. This
> will
> > depend on how accessible the project is for new folks to make meaningful
> > contributions. For a project like ours where the code base has been in
> > development for years, it can be quite daunting for new members to just
> > pick up and make contributions. We need to find ways to make it easier
> for
> > people to do so. Malhar, namely the operator library, is an area where
> > people can contribute without requiring deep knowledge or expertise.
> >
> > We have seen operators take time to mature as evidenced by the road taken
> > by some of our commonly used operators to reach production quality. This
> is
> > due to the fact that apart from the core functionality the operator is
> > trying to implement there are many other aspects to address such as
> > performance, idempotency, processing semantics and scalability. It would
> be
> > difficult even for folks familiar with all these aspects to get
> everything
> > right the first time around and produce comprehensive operators let alone
> > first time contributors. At the same time operators cannot reach this
> > maturity level unless they get used in different scenarios and get a good
> > look at by different people. In maturity I am also including API
> stability.
> >
> > I would like to propose creation of a space inside Malhar, a sub-folder,
> > where contributions can first go in if they are not fully ready and when
> > they mature can be moved out of the sub-folder into an existing module
> or a
> > new module of its own, the package paths can remain the same. The
> > evaluation bar for contributions into this space would be more permissive
> > than it is today, it would require the functionality the operator was
> > developed for be working but will not necessitate that all fault tolerant
> > and scalability aspects be addressed. It will also allow new operators
> that
> > are variations of existing operators till such time as we can determine
> if
> > the new functionality can be subsumed by the original operator or it
> makes
> > sense for the new operator to exist as a separate entity. It will be
> up-to
> > committers and contributors to work together and make the decisions as to
> > whether the individual contributions go into this space or are ready to
> > just go into the regular modules.
> >
> > What does everyone think.
> >
> > Thanks,
> > Pramod
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message