flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gyula Fóra <gyf...@apache.org>
Subject Re: [DISCUSS] Dedicated streaming mode and start scripts
Date Tue, 17 Feb 2015 10:55:15 GMT
+1
Let's do this soon to avoid performance issues for streaming.

On Tue, Feb 17, 2015 at 11:39 AM, Fabian Hueske <fhueske@gmail.com> wrote:

> sounds like a good idea to me.
> +1
>
> 2015-02-17 11:28 GMT+01:00 Stephan Ewen <sewen@apache.org>:
>
> > Hi everyone!
> >
> > What do you think about making the streaming execution mode of the system
> > explicit? That means that people start a Flink cluster explicitly in
> Batch
> > mode or in Streaming mode.
> >
> > The rational behind this idea is that I am not sure how batch and
> streaming
> > clusters are really shared in a meaningful way, since streaming programs
> > basically run forever. There are also further differences:
> >
> >   - Memory Management: Streaming jobs do not use the managed memory
> > currently (see [1] and [2])
> >
> >   - Are streaming jobs inherently single user? Initially, I would say
> yes,
> > because you need to know that you provisioned enough compute power to
> keep
> > up with your ingestion rate and that not some other job starts eating
> > shared resources from you (network / disk)
> >
> >   - High Availability will probably look a bit different for a streaming
> > master and a batch master
> >
> > Once we figured the co-existence between streaming and batch in the same
> > cluster out better, we can remove this separation. This does not affect
> any
> > user programs, only the "ops" of the cluster.
> >
> > Greetings,
> > Stephan
> >
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-1368
> > [2] https://issues.apache.org/jira/browse/FLINK-1323
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message