Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
From: "Mao, Wei" <wei.mao@intel.com>
To: "user@flink.apache.org" <user@flink.apache.org>
Subject: How to get latest offsets with FlinkKafkaConsumer
Thread-Topic: How to get latest offsets with FlinkKafkaConsumer
Thread-Index: AdHvFd8C7nwMGl0bTeuZy/n3L8ZXwA==
Date: Fri, 5 Aug 2016 12:36:01 +0000
Message-ID: <D7747EAE1816444ABBB5EFA0054D5ACF23005469@shsmsx102.ccr.corp.intel.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: multipart/alternative;
	boundary="_000_D7747EAE1816444ABBB5EFA0054D5ACF23005469shsmsx102ccrcor_"
MIME-Version: 1.0
archived-at: Fri, 05 Aug 2016 12:36:34 -0000

--_000_D7747EAE1816444ABBB5EFA0054D5ACF23005469shsmsx102ccrcor_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

I am doing some performance tests with Flink (1.0.3 )+ Kafka (0.8.2.2). And=
 I noticed that when I restarted my Flink application, it reads records sta=
rting from the latest offset that I consumed last time, but not the latest =
offsets of that topic in Kafka.

So Is there any way to make it read from last offsets of broker/MyTopic ins=
tead of consumer/MyTopic in Flink?

Thanks,
William


--_000_D7747EAE1816444ABBB5EFA0054D5ACF23005469shsmsx102ccrcor_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
	{font-family:SimSun;
	panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:"\@SimSun";
	panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:#0563C1;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:#954F72;
	text-decoration:underline;}
span.EmailStyle17
	{mso-style-type:personal;
	font-family:"Calibri",sans-serif;
	color:windowtext;}
span.EmailStyle18
	{mso-style-type:personal-reply;
	font-family:"Calibri",sans-serif;
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-size:10.0pt;}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"#0563C1" vlink=3D"#954F72">
<div class=3D"WordSection1">
<p class=3D"MsoNormal">I am doing some performance test<span style=3D"color=
:#1F497D">s</span> with Flink (1.0.3 )&#43; Kafka (0.8.2.2). And I noticed =
that when I restarted my Flink application, it reads records starting from =
the latest offset that I consumed last time,
 but not the latest offsets of that topic in Kafka. &nbsp;<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">So Is there any way to make it read from last offset=
s of broker/<span style=3D"color:#1F497D">MyT</span>opic instead of consume=
r/<span style=3D"color:#1F497D">MyT</span>opic in Flink? &nbsp;<o:p></o:p><=
/p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
<p class=3D"MsoNormal">Thanks,<o:p></o:p></p>
<p class=3D"MsoNormal">William<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p>&nbsp;</o:p></p>
</div>
</body>
</html>

--_000_D7747EAE1816444ABBB5EFA0054D5ACF23005469shsmsx102ccrcor_--