flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: Fixing the ExecutionConfig
Date Thu, 12 Nov 2015 10:35:11 GMT
+1 for separating concerns by having a StreamExecutionConfig and a
BatchExecutionConfig with inheritance from ExecutionConfig for general
options. Not sure about the pre-flight and runtime options. I think
they are ok in one config.

On Wed, Nov 11, 2015 at 1:24 PM, Robert Metzger <rmetzger@apache.org> wrote:
> I think now (before the 1.0 release) is the right time to clean it up.
>
> Are you suggesting to have two execution configs for batch and streaming?
>
> I'm not sure if we need to distinguish between pre-flight and runtime
> options: From a user's perspective, it doesn't matter. For example the
> serializer settings are evaluated during pre-flight but they have a impact
> during execution.
>
>
>
>
>
>
> On Wed, Nov 11, 2015 at 11:59 AM, Stephan Ewen <sewen@apache.org> wrote:
>
>> Hi all!
>>
>> The ExecutionConfig is a bit of a strange thing right now. It looks like it
>> became the place where everyone just put the stuff they want to somehow
>> push from the client to runtime, plus a random assortment of conflig flags.
>>
>> As a result:
>>
>>   - The ExecutionConfig is available in batch and streaming, but has a
>> number of fields that are very streaming specific, like the watermark
>> interval, etc.
>>
>>   - Several fields that are purely pre-flight time relevant are in there,
>> like whether to use the closure cleaner, or whether to force Avro or Kryo
>> serializers for POJOs.
>>
>> Any interest in cleaning this up? Because these messy classes simply grow
>> ever more messy unless we establish a proper definition of what its
>> concerns and non-concerns are...
>>
>> Greetings,
>> Stephan
>>

Mime
View raw message