Thanks Robert appreciate your help. 

On Fri, May 6, 2016 at 3:07 PM, Robert Metzger <rmetzger@apache.org> wrote:
Hi,

yes, you can use Kafka's configuration setting for that. Its called "auto.offset.reset". Setting it to "latest" will change the restart behavior to the current offset ("earliest" is the opposite).

How heavy is the processing you are doing? 4500 events/second sounds not like a lot of throughput. 

On Wed, May 4, 2016 at 8:24 AM, Balaji Rajagopalan <balaji.rajagopalan@olacabs.com> wrote:
I am using the flink connector to read from a kafka stream, I ran into the problem where the flink job went down due to some application error, it was down for sometime, meanwhile the kafka queue was growing as expected no consumer to consume from the given group , and when I started the flink it started consuming the messages no problem so far, but consumer lag was huge since producer is a fast producer about 4500 events/sec. My question is there any flink connector configuration which can force it read from the latest offset when the flink application starts since in my application logic I do not care about older events. 

balaji