Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <CAK2vteoQbcDX+vfi5Y0t2zjT0rRYVLi2Vi5wyTNt4+gPxn8DZg@mail.gmail.com>
References: <CAC_UMNFGwWMAYJBErqDoaMSPd_3RonznW38Pis6aUVwtnULLTw@mail.gmail.com>
 <CAAdrtT3o3469mWQbNCH+ZMV1O255Y3jpqmTphG9cLzLy0OQEeA@mail.gmail.com> <CAK2vteoQbcDX+vfi5Y0t2zjT0rRYVLi2Vi5wyTNt4+gPxn8DZg@mail.gmail.com>
From: Fabian Hueske <fhueske@gmail.com>
Date: Thu, 27 Oct 2016 09:58:41 +0200
Message-ID: <CAAdrtT3UDeSaaYoMAKAoVZ4OwB0iipEcuHx=6NLJuw-M8ryw-g@mail.gmail.com>
Subject: Re: [Discuss] FLIP-13 Side Outputs in Flink
To: "dev@flink.apache.org" <dev@flink.apache.org>
Content-Type: multipart/alternative; boundary=001a1140e5fef42e5b053fd41dfe
archived-at: Thu, 27 Oct 2016 07:59:20 -0000

--001a1140e5fef42e5b053fd41dfe
Content-Type: text/plain; charset=UTF-8

Hi CPC,

I agree, support for side outputs would be nice for DataSet as well.
However, this is not easily possible because it would require an extensive
rewrite of the DataSet optimizer.
IMO, that's out of scope for this proposal.

Cheers, Fabian

2016-10-27 0:29 GMT+02:00 CPC <achalil@gmail.com>:

> Is it just related to stream api? This feature could be really useful for
> etl scenarios with dataset api as well.
>
> On Oct 26, 2016 22:29, "Fabian Hueske" <fhueske@gmail.com> wrote:
>
> > Hi Chen,
> >
> > thanks for this interesting proposal. I think side output would be a very
> > valuable feature to have!
> >
> > I went of the FLIP and have a few questions.
> >
> > - Will multiple side outputs of the same type be supported?
> > - If I got it right, the FLIP proposes to change the signatures of many
> > user-defined functions (FlatMapFunction, WindowFunction, ...). Most of
> > these interfaces/classes are annotated with @Public, which means we
> cannot
> > change them in the Flink 1.x release line. What would be alternatives? I
> > can think of a) casting the Collector into a RichCollector (as you do in
> > your prototype) or b) retrieve the RichCollector from the RuntimeContext
> > that a RichFunction provides.
> >
> > I'm not so familiar with the internals of the DataStream API, so I leave
> > comments on that to other.
> >
> > Best, Fabian
> >
> > 2016-10-25 18:00 GMT+02:00 Chen Qin <cq@uber.com>:
> >
> > > Hey folks,
> > >
> > > Please give feedback on FLIP-13!
> > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-
> > > 13+Side+Outputs+in+Flink
> > > JIRA task link to google doc
> > > https://issues.apache.org/jira/browse/FLINK-4460
> > >
> > > Thanks,
> > > Chen Qin
> > >
> >
>

--001a1140e5fef42e5b053fd41dfe--