Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
References: 
 <CA+faj9yyae7pSuoNLq3Sn0v__Zmv1dHmXN8f_=dEsFvggW_CGQ@mail.gmail.com>
 <CANMXwW3jR6AhNnj9_47TUjNB91OmyUZ=8yC5LfJQV4Zu+6w-Xw@mail.gmail.com>
In-Reply-To: 
 <CANMXwW3jR6AhNnj9_47TUjNB91OmyUZ=8yC5LfJQV4Zu+6w-Xw@mail.gmail.com>
From: =?UTF-8?Q?Gyula_F=C3=B3ra?= <gyula.fora@gmail.com>
Date: Tue, 07 Jul 2015 14:14:48 +0000
Message-ID: 
 <CA+faj9x=LGjYoOT20Ydv8_uaLquf76kKxNf6Z=j05wDet=ge3Q@mail.gmail.com>
Subject: Re: Rework of streaming iteration API
To: "dev@flink.apache.org" <dev@flink.apache.org>
Content-Type: multipart/alternative; boundary=001a11348408ae5cb9051a49a518

--001a11348408ae5cb9051a49a518
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Sorry Stephan I meant it slightly differently, I see your point:

DataStream source =3D ...
SingleInputOperator mapper =3D source.map(...)
mapper.addInput()

So the add input would be a method of the operator not the stream.

Aljoscha Krettek <aljoscha@apache.org> ezt =C3=ADrta (id=C5=91pont: 2015. j=
=C3=BAl. 7., K,
16:12):

> I think this would be good yes. I was just about to open an Issue for
> changing the Streaming Iteration API. :D
>
> Then we should also make the implementation very straightforward and
> simple, right now, the implementation of the iterations is all over the
> place.
>
> On Tue, 7 Jul 2015 at 15:57 Gyula F=C3=B3ra <gyfora@apache.org> wrote:
>
> > Hey,
> >
> > Along with the suggested changes to the streaming API structure I think
> we
> > should also rework the "iteration" api. Currently the iteration api tri=
es
> > to mimic the syntax of the batch API while the runtime behaviour is qui=
te
> > different.
> >
> > What we create instead of iterations is really just cyclic streams (loo=
ps
> > in the streaming job), so the API should somehow be intuitive about thi=
s
> > behaviour.
> >
> > I suggest to remove the explicit iterate call and instead add a method =
to
> > the StreamOperators that allows to connect feedback inputs (create
> loops).
> > It would look like this:
> >
> > A mapper that does nothing but iterates over some filtered input:
> >
> > *Current API :*
> > DataStream source =3D ..
> > IterativeDataStream it =3D source.iterate()
> > DataStream mapper =3D it.map(noOpMapper)
> > DataStream feedback =3D mapper.filter(...)
> > it.closeWith(feedback)
> >
> > *Suggested API :*
> > DataStream source =3D ..
> > DataStream mapper =3D source.map(noOpMapper)
> > DataStream feedback =3D mapper.filter(...)
> > mapper.addInput(feedback)
> >
> > The suggested approach would let us define inputs to operators after th=
ey
> > are created and implicitly union them with the normal input. This is I
> > think a much clearer approach than what we have now.
> >
> > What do you think?
> >
> > Gyula
> >
>

--001a11348408ae5cb9051a49a518--