flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <mj...@informatik.hu-berlin.de>
Subject Re: Is there Any api that let DataStream join DataSet ?
Date Mon, 29 Jun 2015 09:28:43 GMT
I am wondering what the semantics of a DataStream created from a file
is. It should be a regular (but finite) stream. From my understanding, a
Window-Join is defined with some ts-constraint. So the static file part
will also have this restriction in the join, right? However, a
file-stream-join should join *all* data from the file with each element
in the stream... It seems to me, that a file-DataStream would not yield
this result. Am I wrong?

On 06/29/2015 11:00 AM, Stephan Ewen wrote:
> If you only want to "join" a finite data set (like a file) to a stream, you
> can do that. you can create a DataStream from a (distributed) file.
> If you want specific batch-api operations, this is still on the roadmap,
> not in yet, as Marton said.
> On Sun, Jun 28, 2015 at 10:45 AM, Márton Balassi <balassi.marton@gmail.com>
> wrote:
>> Hi,
>> Flink currently does not have explicit Api support for that, but is
>> definitely possible to do. In fact Gyula (cc-d) mocked up a prototype for a
>> similar problem some time ago.
>> The idea needs some refinement to properly support all the viable use cases
>> though and the streaming Api currently has some more pressing challenges
>> than this integration. :)
>> It's on our roadmap, but is not an immediate task. Could you tell us more
>> about your use case?
>> Best,
>> Marton
>> On Jun 28, 2015 8:29 AM, "马国维" <maguowei@outlook.com> wrote:
>>> Hi,everyone:
>>> Is there Any api that let the DataStream join a DataSet ? I have read all
>>> the document But I can't find .
>>> If Flink now does not have the api, will Flink support it in the future ?
>>>  thanks a lot!

View raw message