apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Yan <da...@datatorrent.com>
Subject Re: A proposal for Malhar
Date Sat, 28 May 2016 02:47:37 GMT
The two ideas are not conflicting, but rather complementing.

On the contrary, putting a new process for people trying to contribute
while NOT addressing the old unused subpar operators in the repository is
what is conflicting.

Keep in mind that when people try to contribute, they always look at the
existing operators already in the repository as examples and likely a model
for their new operators.

David


On Fri, May 27, 2016 at 4:05 PM, Amol Kekre <amol@datatorrent.com> wrote:

> Yes there are two conflicting threads now. The original thread was to open
> up a way for contributors to submit code in a dir (contrib?) as long as
> license part of taken care of.
>
> On the thread of removing non-used operators -> How do we know what is
> being used?
>
> Thks,
> Amol
>
>
> On Fri, May 27, 2016 at 3:40 PM, Sandesh Hegde <sandesh@datatorrent.com>
> wrote:
>
> > +1 for removing the not-used operators.
> >
> > So we are creating a process for operator writers who don't want to
> > understand the platform, yet wants to contribute? How big is that set?
> > If we tell the app-user, here is the code which has not passed all the
> > checklist, will they be ready to use that in production?
> >
> > This thread has 2 conflicting forces, reduce the operators and make it
> easy
> > to add more operators.
> >
> >
> >
> > On Fri, May 27, 2016 at 3:03 PM Pramod Immaneni <pramod@datatorrent.com>
> > wrote:
> >
> > > On Fri, May 27, 2016 at 2:30 PM, Gaurav Gupta <
> gaurav.gopi123@gmail.com>
> > > wrote:
> > >
> > > > Pramod,
> > > >
> > > > By that logic I would say let's put all partitionable operators into
> > one
> > > > folder, non-partitionable operators in another and so on...
> > > >
> > >
> > > Remember the original goal of making it easier for new members to
> > > contribute and managing those contributions to maturity. It is not a
> > > functional level separation.
> > >
> > >
> > > > When I look at hadoop code I see these annotations being used at
> class
> > > > level and not at package/folder level.
> > >
> > >
> > > I had a typo in my email, I meant to say "think of this like a
> folder..."
> > > as an analogy and not literally.
> > >
> > > Thanks
> > >
> > >
> > > > Thanks
> > > >
> > > > On Fri, May 27, 2016 at 2:10 PM, Pramod Immaneni <
> > pramod@datatorrent.com
> > > >
> > > > wrote:
> > > >
> > > > > On Fri, May 27, 2016 at 1:05 PM, Gaurav Gupta <
> > > gaurav.gopi123@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Can same goal not be achieved by
> > > > > > using
> org.apache.hadoop.classification.InterfaceStability.Evolving
> > /
> > > > > > org.apache.hadoop.classification.InterfaceStability.Unstable
> > > > annotation?
> > > > > >
> > > > >
> > > > > I think it is important to localize the additions in one place so
> > that
> > > it
> > > > > becomes clearer to users about the maturity level of these, easier
> > for
> > > > > developers to track them towards the path to maturity and also
> > > provides a
> > > > > clearer directive for committers and contributors on acceptance of
> > new
> > > > > submissions. Relying on the annotations alone makes them spread all
> > > over
> > > > > the place and adds an additional layer of difficulty in
> > identification
> > > > not
> > > > > just for users but also for developers who want to find such
> > operators
> > > > and
> > > > > improve them. This of this like a folder level annotation where
> > > > everything
> > > > > under this folder is unstable or evolving.
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > > >
> > > > > > On Fri, May 27, 2016 at 12:35 PM, David Yan <
> david@datatorrent.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Malhar in its current state, has way too
many operators
> > that
> > > > fall
> > > > > > in
> > > > > > > > the
> > > > > > > > > > "non-production quality" category. We should
make it
> > obvious
> > > to
> > > > > > users
> > > > > > > > > that
> > > > > > > > > > which operators are up to par, and which
operators are
> not,
> > > and
> > > > > > maybe
> > > > > > > > > even
> > > > > > > > > > remove those that are likely not ever used
in a real use
> > > case.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I am ambivalent about revisiting older operators
and doing
> > this
> > > > > > > exercise
> > > > > > > > as
> > > > > > > > > this can cause unnecessary tensions. My original
intent is
> > for
> > > > > > > > > contributions going forward.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > IMO it is important to address this as well. Operators
> outside
> > > the
> > > > > play
> > > > > > > > area should be of well known quality.
> > > > > > > >
> > > > > > > >
> > > > > > > I think this is important, and I don't anticipate much
tension
> if
> > > we
> > > > > > > establish clear criteria.
> > > > > > > It's not helpful if we let the old subpar operators stay
and
> put
> > up
> > > > the
> > > > > > > bars for new operators.
> > > > > > >
> > > > > > > David
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message