flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeyhun Karimov <je.kari...@gmail.com>
Subject Re: Stream conversion
Date Thu, 04 Feb 2016 09:29:38 GMT
Hi Matthias,

This need not to be necessarily in api functions. I just want to get a
roadmap to add this functionality. Should I save each window's data into
disk and create a new dataset environment in parallel? Or change trigger
functionality maybe?

I have large windows. As I asked in previous question, in flink the problem
with large windows (that data inside windows may not fit in memory) will be
solved. So, after getting the data of window, I want to do more than the
functions in stream api. Therefore I need to convert it to dataset. Any
roadmap would be appreciated.

On Thu, Feb 4, 2016 at 10:23 AM Matthias J. Sax <mjsax@apache.org> wrote:

> Hi Sane,
> Currently, DataSet and DataStream API a strictly separated. Thus, this
> is not possible at the moment.
> What kind of operation do you want to perform on the data of a window?
> Why do you want to convert the data into a data set?
> -Matthias
> On 02/04/2016 10:11 AM, Sane Lee wrote:
> > Dear all,
> >
> > I want to convert the data from each window of stream to dataset. What
> > is the best way to do that?  So, while streaming, at the end of each
> > window I want to convert those data to dataset and possible apply
> > dataset transformations to it.
> > Any suggestions?
> >
> > -best
> > -sane

View raw message