Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
From: MAHESH KUMAR <r.mahesh.kumar.blr@gmail.com>
Date: Thu, 9 Mar 2017 14:09:34 -0700
Message-ID: <CANZWe4QHkhcyf5bjcCiBBwhA9TOZRgpT-t5QWjJszLJ4wVsA7w@mail.gmail.com>
Subject: Flink - Writing Test Case for the Datastream
To: user@flink.apache.org
Content-Type: multipart/alternative; boundary=001a1142f506851aa5054a52a998
archived-at: Thu, 09 Mar 2017 21:09:49 -0000

--001a1142f506851aa5054a52a998
Content-Type: text/plain; charset=UTF-8

Hi Team,

I am trying to write test cases to check whether the job is getting
executed as desired. I am using the Flink test util. I am trying to do a
end to end testing where Flink reads from a Kafka Queue, does some
processing and then writes the output to another topic of the Kafka Queue.
My objective is to read the message from the output topic and check if it
has the same message as expected.

I have got Zookeeper and Kafka configured for the test. When I start the
Flink Job, it never terminates since it's source is a Kafka Source. Is
there a way to run a job for a specific interval of time or how do I go
about testing this scenario. Is there any documentation/example for running
test cases such as these?

My code currently looks something like this:

class StreamingMultipleTest extends StreamingMultipleProgramsTestBase
{

@Before def initialize() = {
// Start Kafka, Zookeeper
// Call the run method of the Flink Class - FlinkClass.run()  // This class
contains the env.execute()

// My code does not execute any further since the previous call is never
returned.
}

@Test def Test1() = {
// Check if the Output Topic of the Kafka Queue is as expected -
AssertStatement

}

@After def closeServices() = {
// Stop Zookeeper and Kafka
}

}


Thanks and Regards,
Mahesh

-- 

Mahesh Kumar Ravindranathan
Data Streaming Engineer
Oracle Marketing Cloud - Social Platform
Contact No:+1(720)492-4445

--001a1142f506851aa5054a52a998
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Team,<div><br></div><div>I am trying to write test case=
s to check whether the job is getting executed as desired. I am using the F=
link test util. I am trying to do a end to end testing where Flink reads fr=
om a Kafka Queue, does some processing and then writes the output to anothe=
r topic of the Kafka Queue. My objective is to read the message from the ou=
tput topic and check if it has the same message as expected.<br clear=3D"al=
l"></div><div><br></div><div>I have got Zookeeper and Kafka configured for =
the test. When I start the Flink Job, it never terminates since it&#39;s so=
urce is a Kafka Source. Is there a way to run a job for a specific interval=
 of time or how do I go about testing this scenario. Is there any documenta=
tion/example for running test cases such as these?</div><div><br></div><div=
>My code currently looks something like this:</div><div><br></div><div>clas=
s StreamingMultipleTest extends StreamingMultipleProgramsTestBase=C2=A0</di=
v><div>{<br></div><div><br></div><div>@Before def initialize() =3D {</div><=
div>// Start Kafka, Zookeeper</div><div>// Call the run method of the Flink=
 Class - FlinkClass.run() =C2=A0// This class contains the env.execute()</d=
iv><div><br></div><div>// My code does not execute any further since the pr=
evious call is never returned.</div><div>}</div><div><br></div><div>@Test d=
ef Test1() =3D {</div><div>// Check if the Output Topic of the Kafka Queue =
is as expected - AssertStatement</div><div><br></div><div>}</div><div><br><=
/div><div>@After def closeServices() =3D {</div><div>// Stop Zookeeper and =
Kafka</div><div>}</div><div><br></div><div>}</div><div><br></div><div><br><=
/div><div>Thanks and Regards,</div><div>Mahesh</div><div><br></div>-- <br><=
div class=3D"gmail_signature"><div dir=3D"ltr"><div><div dir=3D"ltr"><div d=
ir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><span style=3D"font-size:12.8p=
x"><br></span></div><div dir=3D"ltr"><span style=3D"font-size:12.8px">Mahes=
h Kumar Ravindranathan =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0=C2=A0</span><div style=3D"font-size:12.8px">Data Streaming Engineer=C2=
=A0</div><div style=3D"font-size:12.8px">Oracle Marketing Cloud - Social Pl=
atform</div><div style=3D"font-size:12.8px">Contact No:<a href=3D"tel:%2B1%=
28720%29492-4445" value=3D"+17204924445" style=3D"color:rgb(17,85,204)" tar=
get=3D"_blank">+1(720)492-4445</a></div><div><br></div></div></div></div></=
div></div></div></div>
</div>

--001a1142f506851aa5054a52a998--