apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Re: A proposal for Malhar
Date Fri, 27 May 2016 23:14:18 GMT
Amol,

I would suggest starting a separate thread for that discussion.

Thanks

On Fri, May 27, 2016 at 4:05 PM, Amol Kekre <amol@datatorrent.com> wrote:

> Yes there are two conflicting threads now. The original thread was to open
> up a way for contributors to submit code in a dir (contrib?) as long as
> license part of taken care of.
>
> On the thread of removing non-used operators -> How do we know what is
> being used?
>
> Thks,
> Amol
>
>
> On Fri, May 27, 2016 at 3:40 PM, Sandesh Hegde <sandesh@datatorrent.com>
> wrote:
>
> > +1 for removing the not-used operators.
> >
> > So we are creating a process for operator writers who don't want to
> > understand the platform, yet wants to contribute? How big is that set?
> > If we tell the app-user, here is the code which has not passed all the
> > checklist, will they be ready to use that in production?
> >
> > This thread has 2 conflicting forces, reduce the operators and make it
> easy
> > to add more operators.
> >
> >
> >
> > On Fri, May 27, 2016 at 3:03 PM Pramod Immaneni <pramod@datatorrent.com>
> > wrote:
> >
> > > On Fri, May 27, 2016 at 2:30 PM, Gaurav Gupta <
> gaurav.gopi123@gmail.com>
> > > wrote:
> > >
> > > > Pramod,
> > > >
> > > > By that logic I would say let's put all partitionable operators into
> > one
> > > > folder, non-partitionable operators in another and so on...
> > > >
> > >
> > > Remember the original goal of making it easier for new members to
> > > contribute and managing those contributions to maturity. It is not a
> > > functional level separation.
> > >
> > >
> > > > When I look at hadoop code I see these annotations being used at
> class
> > > > level and not at package/folder level.
> > >
> > >
> > > I had a typo in my email, I meant to say "think of this like a
> folder..."
> > > as an analogy and not literally.
> > >
> > > Thanks
> > >
> > >
> > > > Thanks
> > > >
> > > > On Fri, May 27, 2016 at 2:10 PM, Pramod Immaneni <
> > pramod@datatorrent.com
> > > >
> > > > wrote:
> > > >
> > > > > On Fri, May 27, 2016 at 1:05 PM, Gaurav Gupta <
> > > gaurav.gopi123@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Can same goal not be achieved by
> > > > > > using
> org.apache.hadoop.classification.InterfaceStability.Evolving
> > /
> > > > > > org.apache.hadoop.classification.InterfaceStability.Unstable
> > > > annotation?
> > > > > >
> > > > >
> > > > > I think it is important to localize the additions in one place so
> > that
> > > it
> > > > > becomes clearer to users about the maturity level of these, easier
> > for
> > > > > developers to track them towards the path to maturity and also
> > > provides a
> > > > > clearer directive for committers and contributors on acceptance of
> > new
> > > > > submissions. Relying on the annotations alone makes them spread all
> > > over
> > > > > the place and adds an additional layer of difficulty in
> > identification
> > > > not
> > > > > just for users but also for developers who want to find such
> > operators
> > > > and
> > > > > improve them. This of this like a folder level annotation where
> > > > everything
> > > > > under this folder is unstable or evolving.
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > > >
> > > > > > On Fri, May 27, 2016 at 12:35 PM, David Yan <
> david@datatorrent.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Malhar in its current state, has way too
many operators
> > that
> > > > fall
> > > > > > in
> > > > > > > > the
> > > > > > > > > > "non-production quality" category. We should
make it
> > obvious
> > > to
> > > > > > users
> > > > > > > > > that
> > > > > > > > > > which operators are up to par, and which
operators are
> not,
> > > and
> > > > > > maybe
> > > > > > > > > even
> > > > > > > > > > remove those that are likely not ever used
in a real use
> > > case.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I am ambivalent about revisiting older operators
and doing
> > this
> > > > > > > exercise
> > > > > > > > as
> > > > > > > > > this can cause unnecessary tensions. My original
intent is
> > for
> > > > > > > > > contributions going forward.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > IMO it is important to address this as well. Operators
> outside
> > > the
> > > > > play
> > > > > > > > area should be of well known quality.
> > > > > > > >
> > > > > > > >
> > > > > > > I think this is important, and I don't anticipate much
tension
> if
> > > we
> > > > > > > establish clear criteria.
> > > > > > > It's not helpful if we let the old subpar operators stay
and
> put
> > up
> > > > the
> > > > > > > bars for new operators.
> > > > > > >
> > > > > > > David
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message