flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Schmidtke <ro.schmid...@gmail.com>
Subject Flink performance pre-packaged vs. self-compiled
Date Wed, 13 Apr 2016 15:03:08 GMT
Hi everyone,

I'm using Flink 0.10.2 for some benchmarks and had to add some small
changes to Flink, which led me to compiling and running it myself. This is
when I noticed a performance difference in the pre-packaged Flink version
that I downloaded from the web (
http://archive.apache.org/dist/flink/flink-0.10.2/flink-0.10.2-bin-hadoop27.tgz)
versus the form of the release-0.10 branch I built myself (mvn
-Dhadoop.version=2.7.1 -Dscala-2.11 -DskipTests -Drat.skip=true clean
install // mvn version 3.0.4).

I ran some version of TeraSort (https://github.com/eastcirclek/terasort)
and I noticed that the pre-packaged version of Flink performs 10-20% better
than the one I built myself (the only tweaks I mead are in the CliFrontend
after the Job has finished running, so I would rule out bad programming on
my side).

Has anyone come across this before? Or could you provide me with clearer
build instructions in order to reproduce the downloadable archive as
closely as possible? Thanks in advance!

Robert

-- 
My GPG Key ID: 336E2680

Mime
View raw message