flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sathi Chowdhury <Sathi.Chowdh...@elliemae.com>
Subject Re: put record to kinesis and then trying consume using flink connector
Date Thu, 27 Apr 2017 04:02:49 GMT
Thanks Alex.
Yes exactly so.I was actually aware of it was challenging to do it in the main method of flink,
even though the push appears after the my  datastream  is attached to kinesis , due to lazy
execution, once the stream is connected then my publish did not work.
If the publish is done by an outside service then obviously no issues..

From: Alex Reid <alex.james.reid@gmail.com>
Date: Tuesday, April 25, 2017 at 4:31 PM
To: Sathi Chowdhury <Sathi.Chowdhury@elliemae.com>
Cc: "user@flink.apache.org" <user@flink.apache.org>
Subject: Re: put record to kinesis and then trying consume using flink connector

Hi Sathi,

I believe the issue is because you pushed the event into the stream and then you started up
a consumer app to start reading after that. If you push an event into the kinesis stream prior
to starting up a reader that sets its initial stream position to LATEST, it will not read
that record because you told it to start reading from the time you started up the consumer
app. "LATEST" does not mean read the last event that was pushed into the stream, it means
start reading from "now"/consumer app start basically.
=============Notice to Recipient: This e-mail transmission, and any documents, files or previous
e-mail messages attached to it may contain information that is confidential or legally privileged,
and intended for the use of the individual or entity named above. If you are not the intended
recipient, or a person responsible for delivering it to the intended recipient, you are hereby
notified that you must not read this transmission and that any disclosure, copying, printing,
distribution or use of any of the information contained in or attached to this transmission
is STRICTLY PROHIBITED. If you have received this transmission in error, please immediately
notify the sender by telephone or return e-mail and delete the original transmission and its
attachments without reading or saving in any manner. Thank you. =============
View raw message