Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <68C660BD-9617-40FC-9C4D-47886CE104CF@yahoo.com>
References: 
 <CAGco--aRBN9aoeoZD6NZ4R2km_o4GyL5NWo-0Q1kk77ap5dr3g@mail.gmail.com>
 <914568111.14811224.1449162664088.JavaMail.yahoo@mail.yahoo.com>
 <68C660BD-9617-40FC-9C4D-47886CE104CF@yahoo.com>
From: Maximilian Michels <mxm@apache.org>
Date: Thu, 3 Dec 2015 19:07:38 +0100
Message-ID: 
 <CAGco--ZPceGF7rTVvGyi0aaJrSPbbuE5xzh8Jmsz4cwx9J48hg@mail.gmail.com>
Subject: Re: Read Kafka topic from the beginning
To: "user@flink.apache.org" <user@flink.apache.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Vladimir,

Did you pass the properties to the FlinkKafkaConsumer?

Cheers,
Max

On Thu, Dec 3, 2015 at 7:06 PM, Vladimir Stoyak <vstoyak@yahoo.com> wrote:
> Gave it a try, but does not seem to help. Is it working for you?
>
> Thanks
>
> Sent from my iPhone
>
>> On Dec 3, 2015, at 6:11 PM, Vladimir Stoyak <vstoyak@yahoo.com> wrote:
>>
>> As far as I know "auto.offset.reset" what to do if offset it not availab=
le or out of bound?
>>
>> Vladimir
>>
>>
>> On Thursday, December 3, 2015 5:58 PM, Maximilian Michels <mxm@apache.or=
g> wrote:
>> Hi Vladimir,
>>
>> You may supply Kafka consumer properties when you create the FlinkKafkaC=
onsumer.
>>
>> Properties props =3D new Properties();
>>
>> // start from largest offset - DEFAULT
>> props.setProperty("auto.offset.reset", "largest");
>> // start from smallest offset
>> props.setProperty("auto.offset.reset", "smallest");
>>
>> I don't think it is possible to start from a specific offset. The
>> offset is only unique per partition. You could modify the offsets in
>> the Zookeeper state but you really have to know what you're doing
>> then.
>>
>> Best regards,
>> Max
>>
>>
>>
>>> On Thu, Dec 3, 2015 at 4:01 PM, Vladimir Stoyak <vstoyak@yahoo.com> wro=
te:
>>> I see that Flink 0.10.1 now supports Keyed Schemas which allows us to r=
ely on Kafka topics set to "compact" retention for data persistence.
>>>
>>> In our topology we wanted to set some topics with Log Compactions enabl=
ed and read topic from the beginning when the topology starts or component =
recovers. Does current Kafka Consumer implementation allow to read all mess=
ages in a topic from the beginning or from a specific offset.
>>>
>>> Thanks,
>>> Vladimir