Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
Date: Wed, 13 Apr 2016 17:03:08 +0200
Message-ID: 
 <CAO2XNQ-R7EdmNa0Vsu5KXv9Mq+9SMYVgjc1jwwnOg8oiy0VGyw@mail.gmail.com>
Subject: Flink performance pre-packaged vs. self-compiled
From: Robert Schmidtke <ro.schmidtke@gmail.com>
To: user@flink.apache.org
Content-Type: multipart/alternative; boundary=001a113de35858738c05305f13a1

--001a113de35858738c05305f13a1
Content-Type: text/plain; charset=UTF-8

Hi everyone,

I'm using Flink 0.10.2 for some benchmarks and had to add some small
changes to Flink, which led me to compiling and running it myself. This is
when I noticed a performance difference in the pre-packaged Flink version
that I downloaded from the web (
http://archive.apache.org/dist/flink/flink-0.10.2/flink-0.10.2-bin-hadoop27.tgz)
versus the form of the release-0.10 branch I built myself (mvn
-Dhadoop.version=2.7.1 -Dscala-2.11 -DskipTests -Drat.skip=true clean
install // mvn version 3.0.4).

I ran some version of TeraSort (https://github.com/eastcirclek/terasort)
and I noticed that the pre-packaged version of Flink performs 10-20% better
than the one I built myself (the only tweaks I mead are in the CliFrontend
after the Job has finished running, so I would rule out bad programming on
my side).

Has anyone come across this before? Or could you provide me with clearer
build instructions in order to reproduce the downloadable archive as
closely as possible? Thanks in advance!

Robert

-- 
My GPG Key ID: 336E2680

--001a113de35858738c05305f13a1
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi everyone,<div><br></div><div>I&#39;m using Flink 0.10.2=
 for some benchmarks and had to add some small changes to Flink, which led =
me to compiling and running it myself. This is when I noticed a performance=
 difference in the pre-packaged Flink version that I downloaded from the we=
b (<a href=3D"http://archive.apache.org/dist/flink/flink-0.10.2/flink-0.10.=
2-bin-hadoop27.tgz">http://archive.apache.org/dist/flink/flink-0.10.2/flink=
-0.10.2-bin-hadoop27.tgz</a>) versus the form of the release-0.10 branch I =
built myself (mvn -Dhadoop.version=3D2.7.1 -Dscala-2.11 -DskipTests -Drat.s=
kip=3Dtrue clean install // mvn version 3.0.4).</div><div><br></div><div>I =
ran some version of TeraSort (<a href=3D"https://github.com/eastcirclek/ter=
asort">https://github.com/eastcirclek/terasort</a>) and I noticed that the =
pre-packaged version of Flink performs 10-20% better than the one I built m=
yself (the only tweaks I mead are in the CliFrontend after the Job has fini=
shed running, so I would rule out bad programming on my side).</div><div><b=
r></div><div>Has anyone come across this before? Or could you provide me wi=
th clearer build instructions in order to reproduce the downloadable archiv=
e as closely as possible? Thanks in advance!</div><div><br></div><div>Rober=
t<br clear=3D"all"><div><br></div>-- <br><div class=3D"gmail_signature"><di=
v dir=3D"ltr">My GPG Key ID: 336E2680<br></div></div>
</div></div>

--001a113de35858738c05305f13a1--