apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amol Kekre <a...@datatorrent.com>
Subject Re: A proposal for Malhar
Date Fri, 27 May 2016 23:30:41 GMT
Agreed

Thks
Amol

On Fri, May 27, 2016 at 4:14 PM, Pramod Immaneni <pramod@datatorrent.com>
wrote:

> Amol,
>
> I would suggest starting a separate thread for that discussion.
>
> Thanks
>
> On Fri, May 27, 2016 at 4:05 PM, Amol Kekre <amol@datatorrent.com> wrote:
>
> > Yes there are two conflicting threads now. The original thread was to
> open
> > up a way for contributors to submit code in a dir (contrib?) as long as
> > license part of taken care of.
> >
> > On the thread of removing non-used operators -> How do we know what is
> > being used?
> >
> > Thks,
> > Amol
> >
> >
> > On Fri, May 27, 2016 at 3:40 PM, Sandesh Hegde <sandesh@datatorrent.com>
> > wrote:
> >
> > > +1 for removing the not-used operators.
> > >
> > > So we are creating a process for operator writers who don't want to
> > > understand the platform, yet wants to contribute? How big is that set?
> > > If we tell the app-user, here is the code which has not passed all the
> > > checklist, will they be ready to use that in production?
> > >
> > > This thread has 2 conflicting forces, reduce the operators and make it
> > easy
> > > to add more operators.
> > >
> > >
> > >
> > > On Fri, May 27, 2016 at 3:03 PM Pramod Immaneni <
> pramod@datatorrent.com>
> > > wrote:
> > >
> > > > On Fri, May 27, 2016 at 2:30 PM, Gaurav Gupta <
> > gaurav.gopi123@gmail.com>
> > > > wrote:
> > > >
> > > > > Pramod,
> > > > >
> > > > > By that logic I would say let's put all partitionable operators
> into
> > > one
> > > > > folder, non-partitionable operators in another and so on...
> > > > >
> > > >
> > > > Remember the original goal of making it easier for new members to
> > > > contribute and managing those contributions to maturity. It is not a
> > > > functional level separation.
> > > >
> > > >
> > > > > When I look at hadoop code I see these annotations being used at
> > class
> > > > > level and not at package/folder level.
> > > >
> > > >
> > > > I had a typo in my email, I meant to say "think of this like a
> > folder..."
> > > > as an analogy and not literally.
> > > >
> > > > Thanks
> > > >
> > > >
> > > > > Thanks
> > > > >
> > > > > On Fri, May 27, 2016 at 2:10 PM, Pramod Immaneni <
> > > pramod@datatorrent.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > On Fri, May 27, 2016 at 1:05 PM, Gaurav Gupta <
> > > > gaurav.gopi123@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Can same goal not be achieved by
> > > > > > > using
> > org.apache.hadoop.classification.InterfaceStability.Evolving
> > > /
> > > > > > > org.apache.hadoop.classification.InterfaceStability.Unstable
> > > > > annotation?
> > > > > > >
> > > > > >
> > > > > > I think it is important to localize the additions in one place
so
> > > that
> > > > it
> > > > > > becomes clearer to users about the maturity level of these,
> easier
> > > for
> > > > > > developers to track them towards the path to maturity and also
> > > > provides a
> > > > > > clearer directive for committers and contributors on acceptance
> of
> > > new
> > > > > > submissions. Relying on the annotations alone makes them spread
> all
> > > > over
> > > > > > the place and adds an additional layer of difficulty in
> > > identification
> > > > > not
> > > > > > just for users but also for developers who want to find such
> > > operators
> > > > > and
> > > > > > improve them. This of this like a folder level annotation where
> > > > > everything
> > > > > > under this folder is unstable or evolving.
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > On Fri, May 27, 2016 at 12:35 PM, David Yan <
> > david@datatorrent.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Malhar in its current state, has way
too many operators
> > > that
> > > > > fall
> > > > > > > in
> > > > > > > > > the
> > > > > > > > > > > "non-production quality" category.
We should make it
> > > obvious
> > > > to
> > > > > > > users
> > > > > > > > > > that
> > > > > > > > > > > which operators are up to par, and
which operators are
> > not,
> > > > and
> > > > > > > maybe
> > > > > > > > > > even
> > > > > > > > > > > remove those that are likely not ever
used in a real
> use
> > > > case.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I am ambivalent about revisiting older operators
and
> doing
> > > this
> > > > > > > > exercise
> > > > > > > > > as
> > > > > > > > > > this can cause unnecessary tensions. My
original intent
> is
> > > for
> > > > > > > > > > contributions going forward.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > IMO it is important to address this as well.
Operators
> > outside
> > > > the
> > > > > > play
> > > > > > > > > area should be of well known quality.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > I think this is important, and I don't anticipate
much
> tension
> > if
> > > > we
> > > > > > > > establish clear criteria.
> > > > > > > > It's not helpful if we let the old subpar operators
stay and
> > put
> > > up
> > > > > the
> > > > > > > > bars for new operators.
> > > > > > > >
> > > > > > > > David
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message