apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Re: Java packages: legacy -> org.apache.apex
Date Wed, 01 Mar 2017 22:24:09 GMT
Inline

On Mon, Feb 27, 2017 at 11:03 PM, Thomas Weise <thw@apache.org> wrote:

> -->
>
> On Mon, Feb 27, 2017 at 1:27 PM, Pramod Immaneni <pramod@datatorrent.com>
> wrote:
>
> > For malhar, for existing operators, I prefer we do this as part of the
> > planned refactoring for breaking the monolith modules into baby packages
> > and would also prefer deprecating the existing operators in place.
>
>
> Refactor into smaller modules was discussed for malhar-contrib and given
> the overall state of that module I think it is OK to defer package renaming
> there. I do however prefer to see the package rename addressed for other
> modules, especially for the main library module.
>

Should we consider breaking the library into smaller modules as well, the
file/block operators for example probably can be in their own module from
just an organizational perspective.


>
>
> > This
> > will help us achieve two things. First, the user will see all the new
> > changes at once as opposed to dealing with it twice (with package rename
> > and dependency changes) and second it will allow for a smoother
> transition
> > as the existing code will still work in a deprecated state. It will also
> > give a more consistent structure to malhar. For new operators, we can go
> > with the new package path but we need to ensure they will get moved into
> > the baby packages as well.
> >
>
> I think existing operators should be renamed so that git history remains. A
> possible solution for backward compatibility could be to subsequently add
> empty subclasses in the previous location (for existing concrete operators
> that we know are actually in use) to simplify migration for users.
>
>
Yes we can do that.


>
> > For demos, we can modify the paths as the apps are typically used
> wholesale
> > and the interface is typically manual interaction.
> >
> > For core, if we are adding new api subsystems, like the launcher api we
> > added recently for example, we can go with new package path but if we are
> > making incremental additions to existing functionality, I feel it is
> better
> > to keep it in the same package. I also prefer we keep the package of the
> > implementation classes consistent with api, for understandability and
> > readability of the code. So, for example, we don't change package path of
> > LogicalPlan as it is an implementation of DAG. It is subjective, but it
> > will be good if we can also do the same with classes closely related to
> the
> > implementation classes as well. Maybe we can moving these on a package by
> > package basis, like everything in com.datatorrent.stram.engine could be
> > moved. For completely internal components like buffer server, we can move
> > them wholesale. We can consider moving all api and classes, when we go to
> > next major release but would like to see if we can find a way to support
> > existing api for one more major release in deprecated mode.
> >
> >
> The point of the major release is to enable backward incompatible changes
> and I don't think it is realistic to support the existing API for another
> major release. IMO it is also not necessary as most existing application
> code refers to operators, attributes and the application interface. Perhaps
> it is possible to keep those around as interface extensions to help
> migration. Custom operators may need to be migrated to reflect API changes,
> and I would consider that a reasonable task for operator developers as part
> of a major upgrade.
>

It would be good if we can keep them as deprecated interface extensions for
one release to provide a smoother transition.


>
> API and implementation in engine are kept separate intentionally. They
> reside in different packages today, so I don't see a problem renaming
> com.datatorrent.stram.engine as you say, even when the API cannot be
> touched right away.
>

They are different packages but sharing a common prefix with api will be
helpful to someone new to codebase in terms of readability. Not a big deal
and can be changed.


>
> Thanks
> >
> > On Mon, Feb 27, 2017 at 7:39 AM, Thomas Weise <thw@apache.org> wrote:
> >
> > > Hi,
> > >
> > > This topic has come up on several PRs and I think it warrants a broader
> > > discussion.
> > >
> > > At the time of incubation, the decision was to defer change of Java
> > > packages from com.datatorrent to org.apache.apex till next major
> release
> > to
> > > ensure backward compatibility for users.
> > >
> > > Unfortunately that has lead to some confusion, as contributors continue
> > to
> > > add new code under legacy packages.
> > >
> > > It is also a wider issue that examples for using Apex continue to refer
> > to
> > > com.datatorrent packages, nearly one year after graduation. More and
> more
> > > user code is being built on top of something that needs to change, the
> > can
> > > is being kicked down the road and users will face more changes later.
> > >
> > > I would like to propose the following:
> > >
> > > 1. All new code has to be submitted under org.apache.apex packages
> > >
> > > 2. Not all code is under backward compatibility restriction and in
> those
> > > cases we can rename the packages right away. Examples: buffer server,
> > > engine, demos/examples, benchmarks
> > >
> > > 3. Discuss when the core API and operators can be changed. For
> operators
> > we
> > > have a bit more freedom to do changes before a major release as most of
> > > them are marked @Evolving and users have the ability to continue using
> > > prior version of Malhar with newer engine due to engine backward
> > > compatibility guarantee.
> > >
> > > Thanks,
> > > Thomas
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message