apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Re: A proposal for Malhar
Date Fri, 27 May 2016 17:19:08 GMT
My comments inline

On Fri, May 27, 2016 at 9:49 AM, Bhupesh Chawda <bhupesh@datatorrent.com>
wrote:

> +1 for the idea of having a separate sub space (or contrib in this case)
> for new contributors to participate. Perhaps the contributor should not be
> given the impression that the task is done unless the operator becomes part
> of some mainstream project like lib.
>

When an operator gets added, with the checklist we can address what is
missing so the contributor will know. It will be upto them whether they
want to continue to improve their contribution or whether they want to
defer to someone else more familiar with platform to take it up to address
the platform capabiltiies. We can also use JIRAs to track these additional
tasks that need to be done.


> Thinking from the perspective of a potential contributor, I think there is
> less motivation in jus developing an operator for its sake. If I were to
> develop some operator, I would first have a usecase in mind. An app, which
> I think would need the operator in question.
> Should these operator contributions be accompanied by an example app,
> demonstrating how to use the operator and perhaps also providing some
> context of the real use case?
>

Examples would be a good way to show a potential use case or serve as a
howto for the operator but we cannot mandate it. We could definitely
recommend it.

Thanks for your input.


>
> ~ Bhupesh
> On 27-May-2016 9:32 am, "Pramod Immaneni" <pramod@datatorrent.com> wrote:
>
> > I think this is a good idea and the committer can help in determining
> this
> > without putting all the burden on the contributor. One way would be to
> list
> > what is missing in terms of platform capabilities and under what
> scenarios
> > the operators cannot be used or unsure whether they will work. As you
> > mentioned it could be informally done in a javadoc or we could introduce
> > special programming language constructs to denote these.
> >
> > On Fri, May 27, 2016 at 8:22 AM, Ganelin, Ilya <
> > Ilya.Ganelin@capitalone.com>
> > wrote:
> >
> > > If we're going to adopt partially completed code, I propose that every
> > new
> > > Malhar operator then contain a checklist as a comment within the class.
> > >
> > > This checklist would be defined by the community and would track the
> > > current development state of the operator. That way there are no
> > unexpected
> > > surprises.
> > >
> > >
> > >
> > > Sent with Good (www.good.com)
> > > ________________________________
> > > From: Amol Kekre <amol@datatorrent.com>
> > > Sent: Friday, May 27, 2016 10:13:06 AM
> > > To: dev@apex.apache.org
> > > Cc: dev@apex.incubator.apache.org
> > > Subject: Re: A proposal for Malhar
> > >
> > > This is a very good idea and will greatly help contributors. The
> > > requirements to submit code to this Malhar folder should be very
> > minimal. A
> > > few that come to my mind
> > >
> > > - Should compile
> > > - License of the external lib (if any) should be Apache compliant
> license
> > >  // Need to see if this is part of ASF guidelines
> > >
> > > Everything else including naming, idempotency, performance, ... should
> be
> > > waived.
> > >
> > > Thks
> > > Amol
> > >
> > >
> > > On Thu, May 26, 2016 at 11:25 PM, Pramod Immaneni <
> > pramod@datatorrent.com>
> > > wrote:
> > >
> > > > As you all know the continued success and growth of an open source
> > > project
> > > > is dependent on new members joining the project and contributing.
> This
> > > will
> > > > depend on how accessible the project is for new folks to make
> > meaningful
> > > > contributions. For a project like ours where the code base has been
> in
> > > > development for years, it can be quite daunting for new members to
> just
> > > > pick up and make contributions. We need to find ways to make it
> easier
> > > for
> > > > people to do so. Malhar, namely the operator library, is an area
> where
> > > > people can contribute without requiring deep knowledge or expertise.
> > > >
> > > > We have seen operators take time to mature as evidenced by the road
> > taken
> > > > by some of our commonly used operators to reach production quality.
> > This
> > > is
> > > > due to the fact that apart from the core functionality the operator
> is
> > > > trying to implement there are many other aspects to address such as
> > > > performance, idempotency, processing semantics and scalability. It
> > would
> > > be
> > > > difficult even for folks familiar with all these aspects to get
> > > everything
> > > > right the first time around and produce comprehensive operators let
> > alone
> > > > first time contributors. At the same time operators cannot reach this
> > > > maturity level unless they get used in different scenarios and get a
> > good
> > > > look at by different people. In maturity I am also including API
> > > stability.
> > > >
> > > > I would like to propose creation of a space inside Malhar, a
> > sub-folder,
> > > > where contributions can first go in if they are not fully ready and
> > when
> > > > they mature can be moved out of the sub-folder into an existing
> module
> > > or a
> > > > new module of its own, the package paths can remain the same. The
> > > > evaluation bar for contributions into this space would be more
> > permissive
> > > > than it is today, it would require the functionality the operator was
> > > > developed for be working but will not necessitate that all fault
> > tolerant
> > > > and scalability aspects be addressed. It will also allow new
> operators
> > > that
> > > > are variations of existing operators till such time as we can
> determine
> > > if
> > > > the new functionality can be subsumed by the original operator or it
> > > makes
> > > > sense for the new operator to exist as a separate entity. It will be
> > > up-to
> > > > committers and contributors to work together and make the decisions
> as
> > to
> > > > whether the individual contributions go into this space or are ready
> to
> > > > just go into the regular modules.
> > > >
> > > > What does everyone think.
> > > >
> > > > Thanks,
> > > > Pramod
> > > >
> > > ________________________________________________________
> > >
> > > The information contained in this e-mail is confidential and/or
> > > proprietary to Capital One and/or its affiliates and may only be used
> > > solely in performance of work or services for Capital One. The
> > information
> > > transmitted herewith is intended only for use by the individual or
> entity
> > > to which it is addressed. If the reader of this message is not the
> > intended
> > > recipient, you are hereby notified that any review, retransmission,
> > > dissemination, distribution, copying or other use of, or taking of any
> > > action in reliance upon this information is strictly prohibited. If you
> > > have received this communication in error, please contact the sender
> and
> > > delete the material from your computer.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message