apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Re: A proposal for Malhar
Date Fri, 27 May 2016 23:12:20 GMT
On Fri, May 27, 2016 at 3:40 PM, Sandesh Hegde <sandesh@datatorrent.com>
wrote:

> +1 for removing the not-used operators.
>
> So we are creating a process for operator writers who don't want to
> understand the platform, yet wants to contribute? How big is that set?
> If we tell the app-user, here is the code which has not passed all the
> checklist, will they be ready to use that in production?
>
> This thread has 2 conflicting forces, reduce the operators and make it easy
> to add more operators.
>

Like I mentioned in my responses to earlier comments on this topic lets
have a separate discussion about existing operators as it requires a
consideration of its own and discussions. This proposal isn't for that.

Thanks


>
>
> On Fri, May 27, 2016 at 3:03 PM Pramod Immaneni <pramod@datatorrent.com>
> wrote:
>
> > On Fri, May 27, 2016 at 2:30 PM, Gaurav Gupta <gaurav.gopi123@gmail.com>
> > wrote:
> >
> > > Pramod,
> > >
> > > By that logic I would say let's put all partitionable operators into
> one
> > > folder, non-partitionable operators in another and so on...
> > >
> >
> > Remember the original goal of making it easier for new members to
> > contribute and managing those contributions to maturity. It is not a
> > functional level separation.
> >
> >
> > > When I look at hadoop code I see these annotations being used at class
> > > level and not at package/folder level.
> >
> >
> > I had a typo in my email, I meant to say "think of this like a folder..."
> > as an analogy and not literally.
> >
> > Thanks
> >
> >
> > > Thanks
> > >
> > > On Fri, May 27, 2016 at 2:10 PM, Pramod Immaneni <
> pramod@datatorrent.com
> > >
> > > wrote:
> > >
> > > > On Fri, May 27, 2016 at 1:05 PM, Gaurav Gupta <
> > gaurav.gopi123@gmail.com>
> > > > wrote:
> > > >
> > > > > Can same goal not be achieved by
> > > > > using org.apache.hadoop.classification.InterfaceStability.Evolving
> /
> > > > > org.apache.hadoop.classification.InterfaceStability.Unstable
> > > annotation?
> > > > >
> > > >
> > > > I think it is important to localize the additions in one place so
> that
> > it
> > > > becomes clearer to users about the maturity level of these, easier
> for
> > > > developers to track them towards the path to maturity and also
> > provides a
> > > > clearer directive for committers and contributors on acceptance of
> new
> > > > submissions. Relying on the annotations alone makes them spread all
> > over
> > > > the place and adds an additional layer of difficulty in
> identification
> > > not
> > > > just for users but also for developers who want to find such
> operators
> > > and
> > > > improve them. This of this like a folder level annotation where
> > > everything
> > > > under this folder is unstable or evolving.
> > > >
> > > > Thanks
> > > >
> > > >
> > > > >
> > > > > On Fri, May 27, 2016 at 12:35 PM, David Yan <david@datatorrent.com
> >
> > > > wrote:
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Malhar in its current state, has way too many
operators
> that
> > > fall
> > > > > in
> > > > > > > the
> > > > > > > > > "non-production quality" category. We should
make it
> obvious
> > to
> > > > > users
> > > > > > > > that
> > > > > > > > > which operators are up to par, and which operators
are not,
> > and
> > > > > maybe
> > > > > > > > even
> > > > > > > > > remove those that are likely not ever used in
a real use
> > case.
> > > > > > > > >
> > > > > > > >
> > > > > > > > I am ambivalent about revisiting older operators and
doing
> this
> > > > > > exercise
> > > > > > > as
> > > > > > > > this can cause unnecessary tensions. My original intent
is
> for
> > > > > > > > contributions going forward.
> > > > > > > >
> > > > > > > >
> > > > > > > IMO it is important to address this as well. Operators
outside
> > the
> > > > play
> > > > > > > area should be of well known quality.
> > > > > > >
> > > > > > >
> > > > > > I think this is important, and I don't anticipate much tension
if
> > we
> > > > > > establish clear criteria.
> > > > > > It's not helpful if we let the old subpar operators stay and
put
> up
> > > the
> > > > > > bars for new operators.
> > > > > >
> > > > > > David
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message