flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Guide/design doc for streaming operator states
Date Thu, 30 Jul 2015 11:46:43 GMT
Ah ok. I think that keyBy() can normally not be chained because we don't
know how the fields in the emitted object change.

On Thu, 30 Jul 2015 at 13:40 Gyula Fóra <gyula.fora@gmail.com> wrote:

> Thanks for the feedback :)
>
> My idea when I wrote that was that you can chain keyBy statements to
> maintain order if your key does not change. Otherwise you are right, we
> need a sorting operator.
>
> Gyula
>
> Aljoscha Krettek <aljoscha@apache.org> ezt írta (időpont: 2015. júl. 30.,
> Cs, 13:18):
>
> > Hi,
> > sorry for the long wait but I finally found the time to read it. It looks
> > good but the later parts of course still need to be fleshed out.
> >
> > I have one comments/questions:
> > In the description of partitioned state you have this sentence:
> "Operations
> > using partitioned state can also benefit from the partial ordering
> > guarantees that the flink runtime provides, to implement deterministic
> > behaviour." How do we provide the ordering guarantees. I would assume
> that
> > after a keyBy() the ordering in each partition is arbitrary, unless we
> add
> > some sorting operator.
> >
> > Cheers,
> > Aljoscha
> >
> > On Thu, 23 Jul 2015 at 15:55 Gyula Fóra <gyfora@apache.org> wrote:
> >
> > > Hey!
> > > I started putting together a guide/design document for the streaming
> > > operator state interfaces and implementations. The idea would be to
> > create
> > > a doc that contains all the details about the implementations so anyone
> > can
> > > use it as a reference later.
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/Stateful+Stream+Processing
> > >
> > > It will probably take me a couple of days to finish it, but in any case
> > > feel free to comment.
> > >
> > > Cheers,
> > > Gyula
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message