flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominique Rondé <dominique.ro...@codecentric.de>
Subject Re: How to start with the first Kafka Message
Date Tue, 22 Mar 2016 10:03:59 GMT
I just place the parameterTool.getProperties() in the 
FlinkKafkaConsumer082 constructor. Now I am able to decide with 
--auto.offset.reset smallest at the commandline each time i start up a 
route.

Thanks for your hints!

Dominique

Am 18.03.2016 um 03:02 schrieb Balaji Rajagopalan:
> If it is a one time thing, you could run the kafka consumer script 
> which has the --from-beginning option, and redirect the output as a 
> socketstream and consume in flink.
>
> On Thu, Mar 17, 2016 at 7:24 PM, Till Rohrmann <trohrmann@apache.org 
> <mailto:trohrmann@apache.org>> wrote:
>
>     Hi Dominique,
>
>     have you tried setting the Kafka property
>     |props.put("auto.offset.reset", "smallest");|?
>
>     Cheers,
>     Till
>
>     ​
>
>     On Thu, Mar 17, 2016 at 1:39 PM, Dominique Rondé
>     <dominique.ronde@codecentric.de
>     <mailto:dominique.ronde@codecentric.de>> wrote:
>
>         Hi folks,
>
>         i have a kafka topic with messages from the last 7 days. Now i
>         have a new flink streaming process and like to consume the
>         messages from the beginning. If I just bring up the topology,
>         the consumer starts from this moment and not from beginning.
>
>         THX
>
>         Dominique
>
>
>

-- 
Dominique Rondé | Senior Consultant

codecentric AG | Kreuznacherstrasse 30 | 60486 Frankfurt | Deutschland
mobil: +49 (0) 172.7182592
www.codecentric.de | blog.codecentric.de | www.meettheexperts.de | www.more4fi.de

Sitz der Gesellschaft: Solingen | HRB 25917| Amtsgericht Wuppertal
Vorstand: Michael Hochgürtel . Mirko Novakovic . Rainer Vehns
Aufsichtsrat: Patric Fedlmeier (Vorsitzender) . Klaus Jäger . Jürgen Schütz


Mime
View raw message