apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Re: Java packages: legacy -> org.apache.apex
Date Sat, 19 Aug 2017 20:42:59 GMT
It will be consistent but there is something to be said about preserving
some history in the present where it allows itself.
On Sat, Aug 19, 2017 at 11:05 AM Vlad Rozov <vrozov@apache.org> wrote:

> Yes, consistency.
>
> Thank you,
>
> Vlad
>
> On 8/19/17 10:57, Pramod Immaneni wrote:
> > Is there any significant gain by doing this?
> >
> > Thanks
> >
> > On Sat, Aug 19, 2017 at 10:36 AM, Vlad Rozov <vrozov@apache.org> wrote:
> >
> >> The parent level pom artifact Id is not referenced by applications, it
> >> exists simply to group modules together under current malhar project.
> The
> >> name can be changed to library-parent(root) or
> apex-library-parent(root) or
> >> something similar. Note that the proposal includes rename of the project
> >> from apex-malhar to apex-library along with the version change.
> >>
> >> Thank you,
> >>
> >> Vlad
> >>
> >>
> >> On 8/19/17 10:10, Pramod Immaneni wrote:
> >>
> >>> Would library module then become apex-library-library? Why change the
> >>> malhar artifact id as the project is called apex-malhar.
> >>>
> >>> Thanks
> >>>
> >>> On Sat, Aug 19, 2017 at 10:03 AM, Vlad Rozov <vrozov@apache.org>
> wrote:
> >>>
> >>> Overall I support the proposal and this is what I was in favor of
> during
> >>>> the last discussion. I'd also propose changing the name of the
> artifact
> >>>> id
> >>>> and git repo from malhar to apex-library as part of the major version
> >>>> change. It will make it more consistent with the entire project and
> will
> >>>> also allow using 1.0 (1.0-SNAPSHOT) version that more closely reflects
> >>>> maturity of the library.
> >>>>
> >>>> Thank you,
> >>>>
> >>>> Vlad
> >>>>
> >>>> On 8/18/17 17:04, Thomas Weise wrote:
> >>>>
> >>>> The proposed change does NOT require changes to existing apps. It
> follows
> >>>>> what you see for example in the JDK where older classes are retired
> >>>>> without
> >>>>> making breaking major release changes.
> >>>>>
> >>>>> Even though not required technically, it looks like the preference
> is to
> >>>>> make this change with a 4.0 release. I will therefore propose to
> change
> >>>>> the
> >>>>> master branch to 4.0-SNAPSHOT and make this and other changes deemed
> >>>>> worthwhile.
> >>>>>
> >>>>> To me it is more important that to newcomers the codebase does not
> look
> >>>>> like an outdated legacy situation and therefore if I have to make
a
> >>>>> choice
> >>>>> than pretty much any version number will do for that. So far I have
> seen
> >>>>> very little interest and initiative in addressing maintenance work
> like
> >>>>> the
> >>>>> packages, checkstyle, CI stability, documentation etc. So it would
be
> >>>>> great
> >>>>> if people interested would step up and contribute.
> >>>>>
> >>>>> Here is the next proposal how to proceed:
> >>>>>
> >>>>>       - create release-3.8 branch for last 3.8 release right away
> >>>>>       - update master to 4.0-SNAPSHOT as part of the open PR #664
> >>>>>       - identify further cleanup work for master
> >>>>>       - release 4.0
> >>>>>
> >>>>> Thanks,
> >>>>> Thomas
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Thu, Aug 17, 2017 at 4:48 PM, Amol Kekre <amol@datatorrent.com>
> >>>>> wrote:
> >>>>>
> >>>>> This following pull request should be taken up in 4.0.0. See my
> comments
> >>>>>
> >>>>>> in
> >>>>>> https://github.com/apache/apex-malhar/pull/664
> >>>>>> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%
> >>>>>> 2Fapache%2Fapex-malhar%2Fpull%2F664&sa=D&ust=1503020269444000&usg=
> >>>>>> AFQjCNHDPZIg2e6I33Jb1XjB5Ir1FkdURQ>
> >>>>>> https://github.com/apache/apex-malhar/pull/662
> >>>>>> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%
> >>>>>> 2Fapache%2Fapex-malhar%2Fpull%2F662&sa=D&ust=1503020269444000&usg=
> >>>>>> AFQjCNF7Xde7X55M3qmc8z-D5y3ZwNg7Fg>
> >>>>>>
> >>>>>> This merge should not be done without a consensus. This will
require
> >>>>>> code
> >>>>>> changes to existing apps.
> >>>>>>
> >>>>>> Thks,
> >>>>>> Amol
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> E:amol@datatorrent.com | M: 510-449-2606 | Twitter: @*amolhkekre*
> >>>>>>
> >>>>>> www.datatorrent.com
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Aug 14, 2017 at 7:48 PM, Thomas Weise <thw@apache.org>
> wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>>> I opened following PRs for the package change:
> >>>>>>>
> >>>>>>> https://github.com/apache/apex-malhar/pull/662
> >>>>>>>
> >>>>>>> Moves all classes with history retained (hence 2 commits).
Also
> >>>>>>> contains
> >>>>>>> checkstyle and other mechanical changes.
> >>>>>>>
> >>>>>>> https://github.com/apache/apex-malhar/pull/664
> >>>>>>>
> >>>>>>> Adds backward compatibility jar.
> >>>>>>>
> >>>>>>> Once above PRs are merged the new artifact can be deployed
and
> >>>>>>> introduced
> >>>>>>> as dependency in malhar-library.
> >>>>>>>
> >>>>>>> Please review.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Thomas
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Sun, Jul 16, 2017 at 7:04 AM, Thomas Weise <thw@apache.org>
> wrote:
> >>>>>>>
> >>>>>>> My original list of work items contained the b/w compatibility
> aspect,
> >>>>>>> I
> >>>>>>> don't think there should be any confusion of whether it
will be
> >>>>>>> covered
> >>>>>>>
> >>>>>>>> here or not.
> >>>>>>>>
> >>>>>>>> The proposed shading will provide the old classes and
they will be
> >>>>>>>>
> >>>>>>>> frozen
> >>>>>>> as of release 3.7. That's the same as making a copy of the
code and
> >>>>>>> never
> >>>>>>> again making changes to the original classes. This cannot
be
> >>>>>>> accomplished
> >>>>>>> by using the older 3.7 release in your project because you
cannot
> use
> >>>>>>> 2
> >>>>>>>
> >>>>>>>> different versions of Malhar in parallel unless you
apply shading.
> >>>>>>>>
> >>>>>>>> The shaded artifact will only expose the com.datatorrent
classes,
> and
> >>>>>>>>
> >>>>>>>> they
> >>>>>>> will be self-contained as the rest of the classes that they
may
> depend
> >>>>>>>> on
> >>>>>>> are shaded. The shaded artifact does not evolve, there are
not more
> >>>>>>> changes
> >>>>>>>
> >>>>>>> to com.datatorrent classes after they are relocated in master.
> >>>>>>>> Thanks,
> >>>>>>>> Thomas
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Sun, Jul 16, 2017 at 2:00 AM, Pramod Immaneni <
> >>>>>>>>
> >>>>>>>> pramod@datatorrent.com
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> I don't think we can limit the topic strictly to relocation
> without
> >>>>>>>> having
> >>>>>>>> a good b/w compatibility story or at least one that
goes far
> enough.
> >>>>>>>>
> >>>>>>>>> The shading idea sounds interesting. Why not let
the shaded
> version
> >>>>>>>>>
> >>>>>>>>> move
> >>>>>>> forward with each release till we hit a major release. If
it is
> going
> >>>>>>>
> >>>>>>>> to
> >>>>>>>>
> >>>>>>> remain pegged at 3.7.0, why shade in the first place as
the regular
> >>>>>>>
> >>>>>>>> 3.7.0
> >>>>>>>> release would do the same job and it would be same as
the loss of
> b/w
> >>>>>>>>
> >>>>>>>>> compatibility with newer releases.
> >>>>>>>>>
> >>>>>>>>> Thanks
> >>>>>>>>>
> >>>>>>>>> On Sat, Jul 15, 2017 at 7:57 AM, Thomas Weise <thw@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Discussing what in the future might become stable
needs to be a
> >>>>>>>>> separate
> >>>>>>>>>
> >>>>>>>> thread, it will be a much bigger discussion.
> >>>>>>>>
> >>>>>>>>> The topic here is to relocate the packages. With
a few exceptions
> >>>>>>>>>> relocation won't affect the semantic versioning.
Semantic
> >>>>>>>>>> versioning
> >>>>>>>>>>
> >>>>>>>>>> is
> >>>>>>>> essentially not effective for Malhar because almost
everything is
> >>>>>>>>
> >>>>>>>>> @Evolving
> >>>>>>>>>
> >>>>>>>>> (and there are reasons for that.. -> separate
topic)
> >>>>>>>>>> I don't really like the idea of creating bw
compatibility stubs
> for
> >>>>>>>>>>
> >>>>>>>>>> the
> >>>>>>>> follow up PR. It creates even more clutter in the source
tree than
> >>>>>>>>
> >>>>>>>>> there
> >>>>>>>>>
> >>>>>>>> already is and so here is an alternative suggestion:
> >>>>>>>>
> >>>>>>>>> https://github.com/tweise/apex-malhar/blob/malhar37-
> >>>>>>>>>> compat/shaded-malhar37/pom.xml
> >>>>>>>>>>
> >>>>>>>>>> Create a shaded artifact that provides the old
com.datatorrent.*
> >>>>>>>>>>
> >>>>>>>>>> classes as
> >>>>>>>>> of release 3.7. Users can include that artifact
if they don't
> want
> >>>>>>>>>> to
> >>>>>>>> change import statements. At the same time they have
an incentive
> to
> >>>>>>>> switch
> >>>>>>>>> to the relocated classes to take advantage of bug
fixes and new
> >>>>>>>>>> functionality.
> >>>>>>>>>>
> >>>>>>>>>> I will work on the first PR that does the relocate.
In the
> >>>>>>>>>> meantime,
> >>>>>>>>>>
> >>>>>>>>>> we
> >>>>>>>> can
> >>>>>>>>
> >>>>>>>>> finalize what backward compatibility support we
want to provide
> and
> >>>>>>>>>> how.
> >>>>>>>> Thanks,
> >>>>>>>>
> >>>>>>>>> Thomas
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Jul 14, 2017 at 11:33 AM, Pramod Immaneni
<
> >>>>>>>>>>
> >>>>>>>>>> pramod@datatorrent.com>
> >>>>>>>>> wrote:
> >>>>>>>>>> How about coming up with a list of @Evolving
operators that we
> >>>>>>>>>> would
> >>>>>>>>>>
> >>>>>>>>> like
> >>>>>>>> to see in the eventual stable list and move those along
with the
> >>>>>>>>>> not
> >>>>>>>>>>
> >>>>>>>>> @Evolving ones in org.apache.apex with b/w stubs
and leave the
> >>>>>>>> rest
> >>>>>>>>> as
> >>>>>>> they
> >>>>>>>>> are. Then have a follow up JIRA for the rest to
be moved over to
> >>>>>>>>>>> contrib
> >>>>>>>>>> and be deprecated.
> >>>>>>>>>>
> >>>>>>>>>>> Thanks
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Jul 14, 2017 at 10:37 AM, Thomas
Weise <
> >>>>>>>>>>>
> >>>>>>>>>>> thomas.weise@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> We need to keep the discussion here on topic,
if other things
> >>>>>>>>>>> are
> >>>>>>>>>>>
> >>>>>>>>>> piled
> >>>>>>>> on
> >>>>>>>>>>> top then nothing gets done.
> >>>>>>>>>>>> Most operators today are not stable,
they are @Evolving. So
> >>>>>>>>>>>>
> >>>>>>>>>>>> perhaps
> >>>>>>>>>> it
> >>>>>>>>> is
> >>>>>>>>>> necessary to have a separate discussion about
that, outside of
> >>>>>>>>>>> this
> >>>>>>>>>>>
> >>>>>>>>>> thread.
> >>>>>>>>> My guess is that there are only a few operators
that we could
> >>>>>>>>>>>> declare
> >>>>>>>>>> stable. Specifically, under contrib the closest
one might have
> >>>>>>>>>>
> >>>>>>>>>>> been
> >>>>>>>>>>>
> >>>>>>>>>> Kafka,
> >>>>>>>>> but that is already superseded by the newer versions.
> >>>>>>>>>>>> Thomas
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Jul 14, 2017 at 10:21 AM, Pramod
Immaneni <
> >>>>>>>>>>>>
> >>>>>>>>>>>> pramod@datatorrent.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>> We had a discussion a while back, agreed
to relegate
> >>>>>>>>>>>> non-stable
> >>>>>>>>>>>>
> >>>>>>>>>>> and
> >>>>>>>> experimental operators to contrib and also added this
to
> >>>>>>>>>>> contribution
> >>>>>>>>>>> guidelines. We aexecuted on this and cleaned
up the repo by
> >>>>>>>>>>> moving
> >>>>>>>>>>> operators deemed non-suitable for production
use at that time
> >>>>>>>>> to
> >>>>>>>>>>> contrib.
> >>>>>>>> So I wouldn't say the operators that are at the top
level
> >>>>>>>>>>>> today
> >>>>>>>>>>>>
> >>>>>>>>>>> or
> >>>>>>> the
> >>>>>>>>> ones
> >>>>>>>>>>>> in library are 0.x.x quality. Granted,
we may need to do one
> >>>>>>>>>>>>> more
> >>>>>>>>>>> pass
> >>>>>>>>> as
> >>>>>>>>>>> some of the operators may no longer be considered
the right
> >>>>>>>>>>>> implementations
> >>>>>>>>>>>>
> >>>>>>>>>>>> with the advent of the windowed operator.
> >>>>>>>>>>>>> Since contrib used to be the place
that housed operators that
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> required
> >>>>>>>>>>> third party libraries, there are some production
quality
> >>>>>>>>>>>
> >>>>>>>>>>>> operators in
> >>>>>>>>>>>>
> >>>>>>>>>>> there
> >>>>>>>>>>> that need to be pulled out into top level
modules.
> >>>>>>>>>>>>> I think we are in agreement that
for operators that we
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> consider
> >>>>>>>>>>> stable,
> >>>>>>>> we
> >>>>>>>>>>>> should provide a b/w stub. I would add
that we strongly
> >>>>>>>>>>>>> consider
> >>>>>>>>>>> creating
> >>>>>>>> the org.apache.apex counterparts of any stable operators
that
> >>>>>>>>>>>> are
> >>>>>>>>>>>>
> >>>>>>>>>>> in
> >>>>>>>>> contrib out in top level modules and have the com.datatorrent
> >>>>>>>>>>> stubs
> >>>>>>>>>>> in
> >>>>>>>>>> contrib.
> >>>>>>>>>>>> For the operators not considered stable,
I would prefer we
> >>>>>>>>>>>>> either
> >>>>>>>>>>> leave
> >>>>>>>>> the
> >>>>>>>>>>>> current package path as is or if we
change the package path
> >>>>>>>>>>>>> then
> >>>>>>>>>>> create
> >>>>>>>> the
> >>>>>>>>>>>> b/w stub as I am not sure how widely
they are in use (so, in
> >>>>>>>>>>>>> essence,
> >>>>>>>>>>> preserve semantic versioning). It would
be good if there is a
> >>>>>>>>>>> followup
> >>>>>>>>>>> JIRA
> >>>>>>>>>>>
> >>>>>>>>>>>> that takes a look at what other operators
can be moved to
> >>>>>>>>>>>>> contrib
> >>>>>>>>>>> with
> >>>>>>>>> the
> >>>>>>>>>>>> advent of the newer frameworks and understanding.
> >>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Jul 14, 2017 at 9:44 AM,
Thomas Weise <
> thw@apache.org
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>> Most of the operators evolve, as is quite
clearly indicated
> >>>>>>>>>>>
> >>>>>>>>>>>> by
> >>>>>>>>>>>> @Evolving
> >>>>>>>> annotations. So while perhaps 0.x.x would be a more
> >>>>>>>>>>>>> appropriate
> >>>>>>>>>>>>>
> >>>>>>>>>>>> version
> >>>>>>>>> number, I don't think you can change that.
> >>>>>>>>>>>>> Thomas
> >>>>>>>>>>>>>> On Fri, Jul 14, 2017 at 9:35
AM, Vlad Rozov <
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> v.rozov@datatorrent.com
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>> If entire library is not stable, its
version should be
> >>>>>>>>>>>>>> 0.x.x
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> and
> >>>>>>>> there
> >>>>>>>>>>> should not be any semantic versioning enabled
or implied.
> >>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> evolves.
> >>>>>>>> If
> >>>>>>>>>>>>> some operators are stable as 3.8.x
version suggests, the
> >>>>>>>>>>>>>> library
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> should
> >>>>>>>>>>> follow semantic versioning and it is not
OK to make a
> >>>>>>>>>>>>>> major
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> change
> >>>>>>>> that
> >>>>>>>>>>>> breaks semantic versioning across the
entire library. It
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> not a
> >>>>>>>> finding
> >>>>>>>>>>> an excuse not to make a change. For me semantic
versioning
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> backward
> >>>>>>>>> compatibility is more important than a proper package
> >>>>>>>>>>>>>> name.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> Thank you,
> >>>>>>>> Vlad
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On 7/14/17 09:11, Thomas
Weise wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Semantic versioning makes
sense when you have a stable
> >>>>>>>>>>>>>>> baseline.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> As
> >>>>>>>>>>> long
> >>>>>>>>>>>>> as
> >>>>>>>>>>>>>>> frequent fixes need to be
made to address basic issues,
> >>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>> makes
> >>>>>>>> no
> >>>>>>>>>>> sense
> >>>>>>>>>>>>> to declare operators stable, which
is why they are marked
> >>>>>>>>>>>>>>> evolving.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Instead of finding reasons to
avoid changes that should
> >>>>>>>>>>>>> have
> >>>>>>>>>>>>>> been
> >>>>>>>>> made a
> >>>>>>>>>>>> long time ago, why not discuss what
a "stable operator"
> >>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> and
> >>>>>>>> which
> >>>>>>>>>>> ones
> >>>>>>>>>>>>> deserve to be in that category.
Those are the ones that
> >>>>>>>>>>>>>>> will
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> get
> >>>>>>>>> the
> >>>>>>>>>>> backward compatibility stub.
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> Thomas
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Fri, Jul 14, 2017
at 8:34 AM, Vlad Rozov <
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> v.rozov@datatorrent.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>> My preference is to agree that
the next Malhar release is
> >>>>>>>>>>>>>>>> 4.0.0
> >>>>>>>>>>>>>> and
> >>>>>>>>>>> make
> >>>>>>>>>>>>> the proposed changes when 3.8.0
goes out. Otherwise why
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> keep
> >>>>>>>> semantic
> >>>>>>>>>>> versioning checks on Malhar in case there
is no version
> >>>>>>>>>>>>>>>> compatibility.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thank you,
> >>>>>>>>>>>>>>> Vlad
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On 7/14/17 08:11,
Thomas Weise wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> next release is
3.8.0
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Fri, Jul 14,
2017 at 7:55 AM, Vlad Rozov <
> >>>>>>>>>>>>>>>>>> v.rozov@datatorrent.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>> next release - 3.9.0 or
4.0.0?
> >>>>>>>>>>>>>>>>>> Thank you,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Vlad
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On 7/13/17
22:25, Thomas Weise wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> It is time
to resurrect this thread and get going with
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> work.
> >>>>>>>>> For the next release, I will sign up to do the
> >>>>>>>>>>>>> package
> >>>>>>>>>>>>>>>>>> move
> >>>>>>>> in
> >>>>>>>>>> Malhar:
> >>>>>>>>>>>> https://issues.apache.org/
> >>>>>>>>>>>>>>>> jira/browse/APEXMALHAR-2517
> >>>>>>>>>>>>>>>>>> In general this
will be straightforward; most classes
> >>>>>>>> in
> >>>>>>>>>>>>>>>>>> Malhar
> >>>>>>>>> are
> >>>>>>>>>>>>> marked
> >>>>>>>>>>>>>>> evolving and it is trivial
for users to change import
> >>>>>>>>>>>>>>>>>>>> statements.
> >>>>>>>>>>>>>>>>>> However,
> >>>>>>>>>>>>>> I would suggest to discuss if
there are selected
> >>>>>>>>>>>>>>>>>>>> operators
> >>>>>>>>>>>>>>>>>> that
> >>>>>>>>>>> are
> >>>>>>>>>>>>> worth
> >>>>>>>>>>>>>>> keeping a backward compatibility
stub in the old
> >>>>>>>>>>>>>>>>>>>> location.
> >>>>>>>>>>>>>>>>>> Here is my plan:
> >>>>>>>>>>> 1. Relocate all classes in *lib* and *contrib*
within
> >>>>>>>>>>>>>>>>>>>> one
> >>>>>>>>>>>>>>>>>> PR -
> >>>>>>>>> this
> >>>>>>>>>>>> is
> >>>>>>>>>>>>>> all
> >>>>>>>>>>>>>>>> IDE automated work
> >>>>>>>>>>>>>>>>>>>> 2. Add
backward compatibility classes, if, any in
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> separate
> >>>>>>>>>>>>>>>>>> PR
> >>>>>>>>>> 3. Create PR for Megh library to reflect moved
> >>>>>>>>>>>> classes
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>> Thomas
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Wed,
Mar 1, 2017 at 2:24 PM, Pramod Immaneni <
> >>>>>>>>>>>>>>>>>>>> pramod@datatorrent.com
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Inline
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Mon,
Feb 27, 2017 at 11:03 PM, Thomas Weise <
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> thw@apache.org>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>> -->
> >>>>>>>>>>>>>>>>>>>>>
On Mon, Feb 27, 2017 at 1:27 PM, Pramod Immaneni <
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
pramod@datatorrent.com
> >>>>>>>>>>>>>>>>>>>>>>
wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
For malhar, for existing operators, I prefer we do
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
this as
> >>>>>>>>>>>>>>>>>>>> part
> >>>>>>>>>>> of
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> planned refactoring for
breaking the monolith
> >>>>>>>>>>>>>>>>>>>>>>
modules
> >>>>>>>>>>>>>>>>>>>> into
> >>>>>>>> baby
> >>>>>>>>>>> packages
> >>>>>>>>>>>>>> and would also prefer deprecating
the existing
> >>>>>>>>>>>>>>>>>>>>>>
operators
> >>>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>> place.
> >>>>>>>>>>>> Refactor into smaller modules was discussed
for
> >>>>>>>>>>>>>>> malhar-contrib
> >>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>> given
> >>>>>>>>>>>>>>> the overall state of that
module I think it is OK
> >>>>>>>>>>>>>>>>>>>>>>
to
> >>>>>>>>>>>>>>>>>>>> defer
> >>>>>>>> package
> >>>>>>>>>>> renaming
> >>>>>>>>>>>>>>> there. I do however prefer
to see the package rename
> >>>>>>>>>>>>>>>>>>>>>
addressed
> >>>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>> other
> >>>>>>>>>>>>>>> modules, especially for
the main library module.
> >>>>>>>>>>>>>>>>>>>>>>
Should we consider breaking the library into
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
smaller
> >>>>>>>>>>>>>>>>>>>> modules
> >>>>>>>> as
> >>>>>>>>>>>> well,
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>
file/block operators for example probably can be in
> >>>>>>>>>>>>>>>>>>>>>
their
> >>>>>>>>>>>>>>>>>>> own
> >>>>>>>>>> module
> >>>>>>>>>>>> from
> >>>>>>>>>>>>>>>> just an organizational
perspective.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
This
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
will help us achieve two things. First, the user
> >>>>>>>>>>>>>>>>>>>>>
will
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> see
> >>>>>>>> all
> >>>>>>>>>>> the
> >>>>>>>>>>>>> new
> >>>>>>>>>>>>>>> changes at once as opposed
to dealing with it
> >>>>>>>>>>>>>>>>>>>>>>>
twice
> >>>>>>>>>>>>>>>>>>>>>
(with
> >>>>>>>> package
> >>>>>>>>>>> rename
> >>>>>>>>>>>>>>> and dependency changes)
and second it will allow
> >>>>>>>>>>>>>>>>>>>>>>
for
> >>>>>>>>>>>>>>>>>>>> a
> >>>>>>> smoother
> >>>>>>>>> transition
> >>>>>>>>>>>>>> as the existing code will still
work in a
> >>>>>>>>>>>>>>>>>>>>>>
deprecated
> >>>>>>>>>>>>>>>>>>>> state.
> >>>>>>>> It
> >>>>>>>>>>> will
> >>>>>>>>>>>>> also
> >>>>>>>>>>>>>>>> give a more consistent
structure to malhar. For new
> >>>>>>>>>>>>>>>>>>>>>>
operators,
> >>>>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>> can
> >>>>>>>>>>>>>>> go
> >>>>>>>>>>>>>>>>>>>>>>
with the new package path but we need to ensure
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
they
> >>>>>>>>>>>>>>>>>>>> will
> >>>>>>>> get
> >>>>>>>>>>> moved
> >>>>>>>>>>>>> into
> >>>>>>>>>>>>>>>> the baby packages as
well.
> >>>>>>>>>>>>>>>>>>>>>>
I think existing operators should be renamed so
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
that
> >>>>>>>>>>>>>>>>>>>> git
> >>>>>>>> history
> >>>>>>>>>>> remains. A
> >>>>>>>>>>>>>> possible solution for backward
compatibility could
> >>>>>>>>>>>>>>>>>>>>>
be
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>> subsequently
> >>>>>>>>> add
> >>>>>>>>>>>>>>>> empty subclasses in
the previous location (for
> >>>>>>>>>>>>>>>>>>>>>>
existing
> >> Thank you,
> >>
> >> Vlad
> >>
>
>
> Thank you,
>
> Vlad
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message