apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Re: Java packages: legacy -> org.apache.apex
Date Sat, 19 Aug 2017 17:10:27 GMT
Would library module then become apex-library-library? Why change the
malhar artifact id as the project is called apex-malhar.

Thanks

On Sat, Aug 19, 2017 at 10:03 AM, Vlad Rozov <vrozov@apache.org> wrote:

> Overall I support the proposal and this is what I was in favor of during
> the last discussion. I'd also propose changing the name of the artifact id
> and git repo from malhar to apex-library as part of the major version
> change. It will make it more consistent with the entire project and will
> also allow using 1.0 (1.0-SNAPSHOT) version that more closely reflects
> maturity of the library.
>
> Thank you,
>
> Vlad
>
> On 8/18/17 17:04, Thomas Weise wrote:
>
>> The proposed change does NOT require changes to existing apps. It follows
>> what you see for example in the JDK where older classes are retired
>> without
>> making breaking major release changes.
>>
>> Even though not required technically, it looks like the preference is to
>> make this change with a 4.0 release. I will therefore propose to change
>> the
>> master branch to 4.0-SNAPSHOT and make this and other changes deemed
>> worthwhile.
>>
>> To me it is more important that to newcomers the codebase does not look
>> like an outdated legacy situation and therefore if I have to make a choice
>> than pretty much any version number will do for that. So far I have seen
>> very little interest and initiative in addressing maintenance work like
>> the
>> packages, checkstyle, CI stability, documentation etc. So it would be
>> great
>> if people interested would step up and contribute.
>>
>> Here is the next proposal how to proceed:
>>
>>     - create release-3.8 branch for last 3.8 release right away
>>     - update master to 4.0-SNAPSHOT as part of the open PR #664
>>     - identify further cleanup work for master
>>     - release 4.0
>>
>> Thanks,
>> Thomas
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Aug 17, 2017 at 4:48 PM, Amol Kekre <amol@datatorrent.com> wrote:
>>
>> This following pull request should be taken up in 4.0.0. See my comments
>>> in
>>> https://github.com/apache/apex-malhar/pull/664
>>> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%
>>> 2Fapache%2Fapex-malhar%2Fpull%2F664&sa=D&ust=1503020269444000&usg=
>>> AFQjCNHDPZIg2e6I33Jb1XjB5Ir1FkdURQ>
>>> https://github.com/apache/apex-malhar/pull/662
>>> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%
>>> 2Fapache%2Fapex-malhar%2Fpull%2F662&sa=D&ust=1503020269444000&usg=
>>> AFQjCNF7Xde7X55M3qmc8z-D5y3ZwNg7Fg>
>>>
>>> This merge should not be done without a consensus. This will require code
>>> changes to existing apps.
>>>
>>> Thks,
>>> Amol
>>>
>>>
>>>
>>> E:amol@datatorrent.com | M: 510-449-2606 | Twitter: @*amolhkekre*
>>>
>>> www.datatorrent.com
>>>
>>>
>>> On Mon, Aug 14, 2017 at 7:48 PM, Thomas Weise <thw@apache.org> wrote:
>>>
>>> Hi,
>>>>
>>>> I opened following PRs for the package change:
>>>>
>>>> https://github.com/apache/apex-malhar/pull/662
>>>>
>>>> Moves all classes with history retained (hence 2 commits). Also contains
>>>> checkstyle and other mechanical changes.
>>>>
>>>> https://github.com/apache/apex-malhar/pull/664
>>>>
>>>> Adds backward compatibility jar.
>>>>
>>>> Once above PRs are merged the new artifact can be deployed and
>>>> introduced
>>>> as dependency in malhar-library.
>>>>
>>>> Please review.
>>>>
>>>> Thanks,
>>>> Thomas
>>>>
>>>>
>>>>
>>>> On Sun, Jul 16, 2017 at 7:04 AM, Thomas Weise <thw@apache.org> wrote:
>>>>
>>>> My original list of work items contained the b/w compatibility aspect,
>>>>>
>>>> I
>>>
>>>> don't think there should be any confusion of whether it will be covered
>>>>> here or not.
>>>>>
>>>>> The proposed shading will provide the old classes and they will be
>>>>>
>>>> frozen
>>>
>>>> as of release 3.7. That's the same as making a copy of the code and
>>>>>
>>>> never
>>>
>>>> again making changes to the original classes. This cannot be
>>>>>
>>>> accomplished
>>>
>>>> by using the older 3.7 release in your project because you cannot use 2
>>>>> different versions of Malhar in parallel unless you apply shading.
>>>>>
>>>>> The shaded artifact will only expose the com.datatorrent classes, and
>>>>>
>>>> they
>>>>
>>>>> will be self-contained as the rest of the classes that they may depend
>>>>>
>>>> on
>>>
>>>> are shaded. The shaded artifact does not evolve, there are not more
>>>>>
>>>> changes
>>>>
>>>>> to com.datatorrent classes after they are relocated in master.
>>>>>
>>>>> Thanks,
>>>>> Thomas
>>>>>
>>>>>
>>>>> On Sun, Jul 16, 2017 at 2:00 AM, Pramod Immaneni <
>>>>>
>>>> pramod@datatorrent.com
>>>
>>>> wrote:
>>>>>
>>>>> I don't think we can limit the topic strictly to relocation without
>>>>>>
>>>>> having
>>>>
>>>>> a good b/w compatibility story or at least one that goes far enough.
>>>>>>
>>>>>> The shading idea sounds interesting. Why not let the shaded version
>>>>>>
>>>>> move
>>>
>>>> forward with each release till we hit a major release. If it is going
>>>>>>
>>>>> to
>>>
>>>> remain pegged at 3.7.0, why shade in the first place as the regular
>>>>>>
>>>>> 3.7.0
>>>>
>>>>> release would do the same job and it would be same as the loss of b/w
>>>>>> compatibility with newer releases.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> On Sat, Jul 15, 2017 at 7:57 AM, Thomas Weise <thw@apache.org>
wrote:
>>>>>>
>>>>>> Discussing what in the future might become stable needs to be a
>>>>>>>
>>>>>> separate
>>>>
>>>>> thread, it will be a much bigger discussion.
>>>>>>>
>>>>>>> The topic here is to relocate the packages. With a few exceptions
>>>>>>> relocation won't affect the semantic versioning. Semantic versioning
>>>>>>>
>>>>>> is
>>>>
>>>>> essentially not effective for Malhar because almost everything is
>>>>>>>
>>>>>> @Evolving
>>>>>>
>>>>>>> (and there are reasons for that.. -> separate topic)
>>>>>>>
>>>>>>> I don't really like the idea of creating bw compatibility stubs
for
>>>>>>>
>>>>>> the
>>>>
>>>>> follow up PR. It creates even more clutter in the source tree than
>>>>>>>
>>>>>> there
>>>>
>>>>> already is and so here is an alternative suggestion:
>>>>>>>
>>>>>>> https://github.com/tweise/apex-malhar/blob/malhar37-
>>>>>>> compat/shaded-malhar37/pom.xml
>>>>>>>
>>>>>>> Create a shaded artifact that provides the old com.datatorrent.*
>>>>>>>
>>>>>> classes as
>>>>>>
>>>>>>> of release 3.7. Users can include that artifact if they don't
want
>>>>>>>
>>>>>> to
>>>
>>>> change import statements. At the same time they have an incentive to
>>>>>>>
>>>>>> switch
>>>>>>
>>>>>>> to the relocated classes to take advantage of bug fixes and new
>>>>>>> functionality.
>>>>>>>
>>>>>>> I will work on the first PR that does the relocate. In the meantime,
>>>>>>>
>>>>>> we
>>>>
>>>>> can
>>>>>>
>>>>>>> finalize what backward compatibility support we want to provide
and
>>>>>>>
>>>>>> how.
>>>>
>>>>> Thanks,
>>>>>>> Thomas
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jul 14, 2017 at 11:33 AM, Pramod Immaneni <
>>>>>>>
>>>>>> pramod@datatorrent.com>
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>> How about coming up with a list of @Evolving operators that we
>>>>>>>>
>>>>>>> would
>>>
>>>> like
>>>>>>
>>>>>>> to see in the eventual stable list and move those along with
the
>>>>>>>>
>>>>>>> not
>>>
>>>> @Evolving ones in org.apache.apex with b/w stubs and leave the
>>>>>>>>
>>>>>>> rest
>>>
>>>> as
>>>>
>>>>> they
>>>>>>>
>>>>>>>> are. Then have a follow up JIRA for the rest to be moved
over to
>>>>>>>>
>>>>>>> contrib
>>>>>>
>>>>>>> and be deprecated.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> On Fri, Jul 14, 2017 at 10:37 AM, Thomas Weise <
>>>>>>>>
>>>>>>> thomas.weise@gmail.com>
>>>>>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> We need to keep the discussion here on topic, if other things
>>>>>>>>>
>>>>>>>> are
>>>
>>>> piled
>>>>>>
>>>>>>> on
>>>>>>>>
>>>>>>>>> top then nothing gets done.
>>>>>>>>>
>>>>>>>>> Most operators today are not stable, they are @Evolving.
So
>>>>>>>>>
>>>>>>>> perhaps
>>>>
>>>>> it
>>>>>>
>>>>>>> is
>>>>>>>
>>>>>>>> necessary to have a separate discussion about that, outside
of
>>>>>>>>>
>>>>>>>> this
>>>>
>>>>> thread.
>>>>>>>>
>>>>>>>>> My guess is that there are only a few operators that
we could
>>>>>>>>>
>>>>>>>> declare
>>>>>>
>>>>>>> stable. Specifically, under contrib the closest one might have
>>>>>>>>>
>>>>>>>> been
>>>>
>>>>> Kafka,
>>>>>>>>
>>>>>>>>> but that is already superseded by the newer versions.
>>>>>>>>>
>>>>>>>>> Thomas
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jul 14, 2017 at 10:21 AM, Pramod Immaneni <
>>>>>>>>>
>>>>>>>> pramod@datatorrent.com>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> We had a discussion a while back, agreed to relegate
>>>>>>>>>>
>>>>>>>>> non-stable
>>>
>>>> and
>>>>>>
>>>>>>> experimental operators to contrib and also added this to
>>>>>>>>>>
>>>>>>>>> contribution
>>>>>>
>>>>>>> guidelines. We aexecuted on this and cleaned up the repo by
>>>>>>>>>>
>>>>>>>>> moving
>>>>
>>>>> operators deemed non-suitable for production use at that time
>>>>>>>>>>
>>>>>>>>> to
>>>
>>>> contrib.
>>>>>>>>
>>>>>>>>> So I wouldn't say the operators that are at the top level
>>>>>>>>>>
>>>>>>>>> today
>>>
>>>> or
>>>>
>>>>> the
>>>>>>>
>>>>>>>> ones
>>>>>>>>>
>>>>>>>>>> in library are 0.x.x quality. Granted, we may need
to do one
>>>>>>>>>>
>>>>>>>>> more
>>>>
>>>>> pass
>>>>>>>
>>>>>>>> as
>>>>>>>>
>>>>>>>>> some of the operators may no longer be considered the
right
>>>>>>>>>>
>>>>>>>>> implementations
>>>>>>>>>
>>>>>>>>>> with the advent of the windowed operator.
>>>>>>>>>>
>>>>>>>>>> Since contrib used to be the place that housed operators
that
>>>>>>>>>>
>>>>>>>>> required
>>>>>>>
>>>>>>>> third party libraries, there are some production quality
>>>>>>>>>>
>>>>>>>>> operators in
>>>>>>
>>>>>>> there
>>>>>>>>>
>>>>>>>>>> that need to be pulled out into top level modules.
>>>>>>>>>>
>>>>>>>>>> I think we are in agreement that for operators that
we
>>>>>>>>>>
>>>>>>>>> consider
>>>
>>>> stable,
>>>>>>>
>>>>>>>> we
>>>>>>>>>
>>>>>>>>>> should provide a b/w stub. I would add that we strongly
>>>>>>>>>>
>>>>>>>>> consider
>>>
>>>> creating
>>>>>>>>
>>>>>>>>> the org.apache.apex counterparts of any stable operators
that
>>>>>>>>>>
>>>>>>>>> are
>>>>
>>>>> in
>>>>>>
>>>>>>> contrib out in top level modules and have the com.datatorrent
>>>>>>>>>>
>>>>>>>>> stubs
>>>>>>
>>>>>>> in
>>>>>>>
>>>>>>>> contrib.
>>>>>>>>>>
>>>>>>>>>> For the operators not considered stable, I would
prefer we
>>>>>>>>>>
>>>>>>>>> either
>>>>
>>>>> leave
>>>>>>>
>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> current package path as is or if we change the package
path
>>>>>>>>>>
>>>>>>>>> then
>>>
>>>> create
>>>>>>>
>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> b/w stub as I am not sure how widely they are in
use (so, in
>>>>>>>>>>
>>>>>>>>> essence,
>>>>>>
>>>>>>> preserve semantic versioning). It would be good if there is a
>>>>>>>>>>
>>>>>>>>> followup
>>>>>>>
>>>>>>>> JIRA
>>>>>>>>>
>>>>>>>>>> that takes a look at what other operators can be
moved to
>>>>>>>>>>
>>>>>>>>> contrib
>>>>
>>>>> with
>>>>>>>
>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> advent of the newer frameworks and understanding.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>> On Fri, Jul 14, 2017 at 9:44 AM, Thomas Weise <thw@apache.org
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>
>>>>>>>> Most of the operators evolve, as is quite clearly indicated
>>>>>>>>>>>
>>>>>>>>>> by
>>>
>>>> @Evolving
>>>>>>>>>
>>>>>>>>>> annotations. So while perhaps 0.x.x would be a more
>>>>>>>>>>>
>>>>>>>>>> appropriate
>>>>
>>>>> version
>>>>>>>>
>>>>>>>>> number, I don't think you can change that.
>>>>>>>>>>>
>>>>>>>>>>> Thomas
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jul 14, 2017 at 9:35 AM, Vlad Rozov <
>>>>>>>>>>>
>>>>>>>>>> v.rozov@datatorrent.com
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> If entire library is not stable, its version
should be
>>>>>>>>>>>>
>>>>>>>>>>> 0.x.x
>>>
>>>> and
>>>>>>
>>>>>>> there
>>>>>>>>>
>>>>>>>>>> should not be any semantic versioning enabled or
implied.
>>>>>>>>>>>>
>>>>>>>>>>> It
>>>
>>>> evolves.
>>>>>>>>
>>>>>>>>> If
>>>>>>>>>>
>>>>>>>>>>> some operators are stable as 3.8.x version suggests,
the
>>>>>>>>>>>>
>>>>>>>>>>> library
>>>>>>
>>>>>>> should
>>>>>>>>>
>>>>>>>>>> follow semantic versioning and it is not OK to make
a
>>>>>>>>>>>>
>>>>>>>>>>> major
>>>
>>>> change
>>>>>>>
>>>>>>>> that
>>>>>>>>>
>>>>>>>>>> breaks semantic versioning across the entire library.
It
>>>>>>>>>>>>
>>>>>>>>>>> is
>>>
>>>> not a
>>>>>>
>>>>>>> finding
>>>>>>>>>>
>>>>>>>>>>> an excuse not to make a change. For me semantic
versioning
>>>>>>>>>>>>
>>>>>>>>>>> and
>>>>
>>>>> backward
>>>>>>>>>
>>>>>>>>>> compatibility is more important than a proper package
>>>>>>>>>>>>
>>>>>>>>>>> name.
>>>
>>>> Thank you,
>>>>>>>>>>>>
>>>>>>>>>>>> Vlad
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 7/14/17 09:11, Thomas Weise wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Semantic versioning makes sense when you
have a stable
>>>>>>>>>>>>>
>>>>>>>>>>>> baseline.
>>>>>>
>>>>>>> As
>>>>>>>>
>>>>>>>>> long
>>>>>>>>>>
>>>>>>>>>>> as
>>>>>>>>>>>>> frequent fixes need to be made to address
basic issues,
>>>>>>>>>>>>>
>>>>>>>>>>>> it
>>>
>>>> makes
>>>>>>
>>>>>>> no
>>>>>>>>
>>>>>>>>> sense
>>>>>>>>>>>
>>>>>>>>>>>> to declare operators stable, which is why
they are marked
>>>>>>>>>>>>>
>>>>>>>>>>>> evolving.
>>>>>>>>
>>>>>>>>> Instead of finding reasons to avoid changes that should
>>>>>>>>>>>>>
>>>>>>>>>>>> have
>>>>
>>>>> been
>>>>>>>
>>>>>>>> made a
>>>>>>>>>>
>>>>>>>>>>> long time ago, why not discuss what a "stable
operator"
>>>>>>>>>>>>>
>>>>>>>>>>>> is
>>>
>>>> and
>>>>>>
>>>>>>> which
>>>>>>>>
>>>>>>>>> ones
>>>>>>>>>>>
>>>>>>>>>>>> deserve to be in that category. Those are
the ones that
>>>>>>>>>>>>>
>>>>>>>>>>>> will
>>>>
>>>>> get
>>>>>>
>>>>>>> the
>>>>>>>>
>>>>>>>>> backward compatibility stub.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Thomas
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jul 14, 2017 at 8:34 AM, Vlad
Rozov <
>>>>>>>>>>>>>
>>>>>>>>>>>> v.rozov@datatorrent.com>
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> My preference is to agree that the next
Malhar release is
>>>>>>>>>>>>>
>>>>>>>>>>>> 4.0.0
>>>>>>
>>>>>>> and
>>>>>>>>
>>>>>>>>> make
>>>>>>>>>>
>>>>>>>>>>> the proposed changes when 3.8.0 goes out. Otherwise
why
>>>>>>>>>>>>>>
>>>>>>>>>>>>> to
>>>
>>>> keep
>>>>>>
>>>>>>> semantic
>>>>>>>>>>>
>>>>>>>>>>>> versioning checks on Malhar in case there
is no version
>>>>>>>>>>>>>>
>>>>>>>>>>>>> compatibility.
>>>>>>>>>>
>>>>>>>>>>> Thank you,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Vlad
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 7/14/17 08:11, Thomas Weise wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> next release is 3.8.0
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Jul 14, 2017 at 7:55
AM, Vlad Rozov <
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> v.rozov@datatorrent.com>
>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> next release - 3.9.0 or 4.0.0?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Vlad
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 7/13/17 22:25, Thomas
Weise wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It is time to resurrect this
thread and get going with
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> the
>>>>
>>>>> work.
>>>>>>>>
>>>>>>>>> For the next release, I will sign up to do the
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> package
>>>
>>>> move
>>>>>>
>>>>>>> in
>>>>>>>
>>>>>>>> Malhar:
>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> jira/browse/APEXMALHAR-2517
>>>
>>>> In general this will be straightforward; most classes
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> in
>>>>
>>>>> Malhar
>>>>>>>>
>>>>>>>>> are
>>>>>>>>>>
>>>>>>>>>>> marked
>>>>>>>>>>>>>>>>> evolving and it is trivial
for users to change import
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> statements.
>>>>>>>>>
>>>>>>>>>> However,
>>>>>>>>>>>>>>>>> I would suggest to discuss
if there are selected
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> operators
>>>>>>
>>>>>>> that
>>>>>>>>
>>>>>>>>> are
>>>>>>>>>>
>>>>>>>>>>> worth
>>>>>>>>>>>>>>>>> keeping a backward compatibility
stub in the old
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> location.
>>>>>>
>>>>>>> Here is my plan:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1. Relocate all classes
in *lib* and *contrib* within
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> one
>>>>
>>>>> PR -
>>>>>>>
>>>>>>>> this
>>>>>>>>>>
>>>>>>>>>>> is
>>>>>>>>>>>
>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> IDE automated work
>>>>>>>>>>>>>>>>> 2. Add backward compatibility
classes, if, any in
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> separate
>>>>>>
>>>>>>> PR
>>>>>>>
>>>>>>>> 3. Create PR for Megh library to reflect moved
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> classes
>>>
>>>> Thanks,
>>>>>>>>>>>>>>>>> Thomas
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Mar 1, 2017 at
2:24 PM, Pramod Immaneni <
>>>>>>>>>>>>>>>>> pramod@datatorrent.com
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Inline
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Feb 27, 2017
at 11:03 PM, Thomas Weise <
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> thw@apache.org>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> -->
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Feb 27, 2017
at 1:27 PM, Pramod Immaneni <
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> pramod@datatorrent.com
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> For malhar, for
existing operators, I prefer we do
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> this as
>>>>>>
>>>>>>> part
>>>>>>>>>
>>>>>>>>>> of
>>>>>>>>>>
>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> planned refactoring
for breaking the monolith
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> modules
>>>
>>>> into
>>>>>>
>>>>>>> baby
>>>>>>>>>
>>>>>>>>>> packages
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> and would
also prefer deprecating the existing
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> operators
>>>>>>
>>>>>>> in
>>>>>>>
>>>>>>>> place.
>>>>>>>>>>
>>>>>>>>>>> Refactor into smaller modules was discussed for
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> malhar-contrib
>>>>>>>>
>>>>>>>>> and
>>>>>>>>>>
>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>> the overall state
of that module I think it is OK
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> to
>>>
>>>> defer
>>>>>>
>>>>>>> package
>>>>>>>>>>
>>>>>>>>>>> renaming
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> there. I do however
prefer to see the package rename
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> addressed
>>>>>>>>
>>>>>>>>> for
>>>>>>>>>>
>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>> modules, especially
for the main library module.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Should we consider
breaking the library into
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> smaller
>>>
>>>> modules
>>>>>>>
>>>>>>>> as
>>>>>>>>>
>>>>>>>>>> well,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> file/block operators
for example probably can be in
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> their
>>>>>>
>>>>>>> own
>>>>>>>
>>>>>>>> module
>>>>>>>>>>>
>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>> just an organizational
perspective.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> will help us achieve
two things. First, the user
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> will
>>>
>>>> see
>>>>>>
>>>>>>> all
>>>>>>>>
>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>>> changes at
once as opposed to dealing with it
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> twice
>>>
>>>> (with
>>>>>>
>>>>>>> package
>>>>>>>>>>
>>>>>>>>>>> rename
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> and dependency
changes) and second it will allow
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> for
>>>
>>>> a
>>>>
>>>>> smoother
>>>>>>>>>
>>>>>>>>>> transition
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> as the existing
code will still work in a
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> deprecated
>>>
>>>> state.
>>>>>>>
>>>>>>>> It
>>>>>>>>
>>>>>>>>> will
>>>>>>>>>>>
>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> give a more
consistent structure to malhar. For new
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> operators,
>>>>>>>>
>>>>>>>>> we
>>>>>>>>>>
>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>> go
>>>>>>>>>>>>>>>>>>> with the new
package path but we need to ensure
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> they
>>>
>>>> will
>>>>>>
>>>>>>> get
>>>>>>>>
>>>>>>>>> moved
>>>>>>>>>>>
>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>> the baby packages
as well.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I think existing
operators should be renamed so
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> that
>>>
>>>> git
>>>>>>
>>>>>>> history
>>>>>>>>>
>>>>>>>>>> remains. A
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> possible solution
for backward compatibility could
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> be
>>>
>>>> to
>>>>
>>>>> subsequently
>>>>>>>>>>>
>>>>>>>>>>>> add
>>>>>>>>>>>>>>>>>>> empty subclasses
in the previous location (for
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> existing
>>>
>>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message