flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henry Saputra <henry.sapu...@gmail.com>
Subject Re: Tests for the Steaming classes
Date Mon, 23 Mar 2015 16:20:53 GMT
+1 for more tests and plus more documentation both internal and javadoc.
Some stuff like memory usafe and data flows are bit different from batch
which have not been documented much.

Looks like more will be coming to Flink to check out streaming solution in
Flink as alternative to others.

- Henry

On Monday, March 23, 2015, Stephan Ewen <sewen@apache.org> wrote:

> Hi everyone!
>
> With the streaming stuff getting heavier exposure, I think it needs a few
> more tests. With so many changes, untested features are running a high risk
> of being "patched away" by accident.
>
> For the runtime and batch API part, we go with the policy that every new
> feature can only be merged is properly backed by tests. The streaming API
> should now follow the same paradigm, in my opinion.
>
> Here are some tests that I suggest to add
>
> 1) Settings/Configuration properly forwarded from ExecutionEnvironment to
> JobGraph
>
> 2) Isolated tests for heavy and critical utilities (like barrier buffer)
>
> 3) Tests on the behavior of the abstract streaming vertex. Checks that
>    - RuntimeContext is porperly initialized
>    - open() and close() is always called (on RichFunctions)
>    - cancelling
>    - close() is called when cancelling a job()
>    - forwarding of barriers
>
> 4) Tests for the JobGraph construction
>    - parallelism properly configured
>    - connections and partitioners properly set
>
> 5) Tests for the chaining construction
>    - functions that can be chained and that cannot be chained
>    - chaining should not be affected by parallelism settings
>
> Do you agree? Should I open a series of JIRAs for this?
>
> Greetings,
> Stephan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message