flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: Restructuring the maven projects
Date Tue, 17 Mar 2015 13:21:02 GMT
+1 I like the proposed structure.

The only thing I was wondering about is whether to name "core" => "batch"?

On Tue, Mar 17, 2015 at 11:37 AM, Márton Balassi <balassi.marton@gmail.com>
wrote:

> +1 for the proposed structure.
>
> I have no explicit preference for having batch and streaming scala together
> or separated. That said streaming scala is considerably thin, it does not
> really require an own maven submodule.
>
> Marked an older JIRA for the same issue as duplicate. [1]
>
> [1] https://issues.apache.org/jira/browse/FLINK-1340
>
> On Tue, Mar 17, 2015 at 10:46 AM, Stephan Ewen <sewen@apache.org> wrote:
>
> > To not let this discussion die, here is a concrete JIRA and a proposed
> > layout to restructure to.
> >
> > What remains to be discusses is whether we want to keep the Scala/Java
> APIs
> > for batch/streaming in separate projects or in one project.
> >
> > Also, we need to find a good time to do this, when we are low on pull
> > requests...
> >
> > https://issues.apache.org/jira/browse/FLINK-1712
> >
> > Proposed Layout:
> >
> >  - flink-hadoop (shaded fat jar)
> >
> >  - Core (Core and Java and Scala)
> >  - Streaming (core + java + scala)
> >  - Runtime
> >  - Client (Client + Optimizer)
> >
> >  - Examples (Java + Scala + Streaming Java + Streaming Scala)
> >  - Tests (test-utils (compile) and tests (test))
> >
> >  - Quickstarts
> >    - Quickstart Java
> >    - Quickstart Scala
> >
> >  - connectors / Input/Output Formats
> >    - Avro
> >    - HBase
> >    - HadoopCompartibility
> >    - HCatalogue
> >    - JDBC
> >    - kafka
> >    - rabbit
> >    - ...
> >
> >  - staging
> >    - Gelly
> >    - Gilbert (ML)
> >    - spargel (deprecated)
> >    - expression API
> >
> >  - contrib
> >
> >  - yarn
> >
> >  - dist
> >
> >  - yarn tests
> >
> >  - java 8
> >
> > On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra <henry.saputra@gmail.com>
> > wrote:
> >
> > > Thanks Marton, having 2 threads discussing same thing can be confusing.
> > >
> > > - Henry
> > >
> > > On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi <mbalassi@apache.org>
> > > wrote:
> > > > Let us consider this thread the standard for the restructure, it is
> > > > perfectly in line with the wishes I have posted.
> > > >
> > > > +1 for keeping the 'flink-' prefix.
> > > >
> > > > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <
> > henry.saputra@gmail.com>
> > > > wrote:
> > > >
> > > >> @Robert, and @Stephan, sure I am ok with it, thanks for the
> responses.
> > > >>
> > > >> - Henry
> > > >>
> > > >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <sewen@apache.org>
> > wrote:
> > > >> > I think this works well together with Marton's restructuring.
> > > >> >
> > > >> > I would vote to keep the "flink-" prefix, because it guarantees
> that
> > > the
> > > >> > produced jars are prefixed with "flink-". Otherwise, we will
have
> to
> > > >> start
> > > >> > configuring a lot...
> > > >> >
> > > >> > Greetings,
> > > >> > Stephan
> > > >> >
> > > >> >
> > > >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <
> > > henry.saputra@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> >> Will this conflict with Marton's restructuring proposal which
> > happens
> > > >> >> in another thread (see "Project restructure" thread in the
dev@
> > > list).
> > > >> >>
> > > >> >> Since we are doing refactoring, may I suggest that we also
remove
> > > >> >> "flink-" prefix since maven group name will indicate it is
part
> of
> > > >> >> Flink.
> > > >> >>
> > > >> >> - Henry
> > > >> >>
> > > >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <sewen@apache.org>
> > > wrote:
> > > >> >> > Hi everyone!
> > > >> >> >
> > > >> >> > I think that by now, quite a bit of the maven project
structure
> > > can be
> > > >> >> > improved to get rid of some legacy artifacts. Especially
the
> > > >> >> "flink-addons"
> > > >> >> > project seems to be a catch-all place for various projects.
> > > >> >> >
> > > >> >> > Here is a suggestion what we could do:
> > > >> >> >
> > > >> >> > 1) Move "flink-yarn" to the root.
> > > >> >> >
> > > >> >> > 2) Move "flink-streaming" to the root (this is planned
anyways
> > for
> > > the
> > > >> >> next
> > > >> >> > release)
> > > >> >> >
> > > >> >> > 3) Create a project "flink-connectors", which will contain
> > "avro",
> > > >> >> "jdbc",
> > > >> >> > and "hbase". Should we have them as separate sub-projects,
or
> as
> > > one
> > > >> >> > project?
> > > >> >> >
> > > >> >> > 4) Consolidate the examples into a single project
> > "flink-examples",
> > > >> where
> > > >> >> > Java, Scala, Streaming examples exist in different packages.
> > > >> >> >
> > > >> >> >
> > > >> >> > Greetings,
> > > >> >> > Stephan
> > > >> >>
> > > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message