flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Pulling Streaming out of staging and project restructure
Date Thu, 01 Oct 2015 10:44:50 GMT
I like it in general. But while we're at it, what is the purpose of the 
flink-tests project, or rather which tests belong there instead of the 
individual projects?

Where would new projects reside in, that previously would have been put 
into flink-staging?

Lastly, I'd like to merge flink-language-binding into flink-python. I 
can go more into detail but the gist of it is that the abstraction just 
doesn't work.

On 01.10.2015 12:40, Márton Balassi wrote:
> Great to see streaming graduating. :)
>
> I like the outline, both getting rid of staging, having the examples
> together and generally flattening the structure are very reasonable to me.
>
> You have listed flink-streaming-examples under flink-streaming-connectors
> and left out some less prominent maven modules, but I assume the first is
> accidental while the second is intentional to make the list a bit briefer.
>
> Best,
>
> Marton
>
>
> On Thu, Oct 1, 2015 at 12:25 PM, Stephan Ewen <sewen@apache.org> wrote:
>
>> Hi all!
>>
>> We are making good headway with reworking the last parts of the Window API.
>> After that, the streaming API should be good to be pulled out of staging.
>>
>> Since we are reorganizing the projects as part of that, I would shift a bit
>> more to bring things a bit more up to date.
>>
>> In this restructure, I would like to get rid of the "flink-staging"
>> project. Anyone who only uses the maven artifacts sees no difference
>> whether a project is in "staging" or not, so it does not help much to have
>> that directory structure.
>> On the other hand, projects have a tendency to linger in staging forever
>> (like avro, spargel, hbase, jdbc, ...)
>>
>> The new structure could be
>>
>> flink-core
>> flink-java
>> flink-scala
>> flink-streaming-core
>> flink-streaming-scala
>>
>> flink-runtime
>> flink-runtime-web
>> flink-optimizer
>> flink-clients
>>
>> flink-shaded
>>    -> flink-shaded-hadoop
>>    -> flink-shaded-hadoop2
>>    -> flink-shaded-include-yarn-tests
>>    -> flink-shaded-curator
>>
>> flink-examples
>>    -> (have all examples, Scala and Java, Batch and Streaming)
>>
>> flink-batch-connectors
>>    -> flink-avro
>>    -> flink-jdbc
>>    -> flink-hadoop-compatibility
>>    -> flink-hbase
>>    -> flink-hcatalog
>>
>> flink-streaming-connectors
>>    -> flink-connector-twitter
>>    -> flink-streaming-examples
>>    -> flink-connector-flume
>>    -> flink-connector-kafka
>>    -> flink-connector-elasticsearch
>>    -> flink-connector-rabbitmq
>>    -> flink-connector-filesystem
>>
>> flink-libraries
>>    -> flink-gelly
>>    -> flink-gelly-scala
>>    -> flink-ml
>>    -> flink-table
>>    -> flink-language-binding
>>    -> flink-python
>>
>>
>> flink-scala-shell
>>
>> flink-test-utils
>> flink-tests
>> flink-fs-tests
>>
>> flink-contrib
>>    -> flink-storm-compatibility
>>    -> flink-storm-compatibility-examples
>>    -> flink-streaming-utils
>>    -> flink-tweet-inputformat
>>    -> flink-operator-stats
>>    -> flink-tez
>>
>> flink-quickstart
>>    -> flink-quickstart-java
>>    -> flink-quickstart-scala
>>    -> flink-tez-quickstart
>>
>> flink-yarn
>> flink-yarn-tests
>>
>> flink-dist
>>
>> flink-benchmark
>>
>>
>> Let me know if that makes sense!
>>
>> Greetings,
>> Stephan
>>


Mime
View raw message