flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "xuhong (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Deleted] (FLINK-1476) Flink VS Spark on loop test
Date Sat, 07 Feb 2015 06:51:36 GMT

     [ https://issues.apache.org/jira/browse/FLINK-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

xuhong updated FLINK-1476:
    Comment: was deleted

(was: Hi Fabian,
   I am very grateful for you advise. By your advise, is that mean i should set the taskmanager.heap.mb
a lager value? I will try it, thank you!)

> Flink VS Spark on loop test
> ---------------------------
>                 Key: FLINK-1476
>                 URL: https://issues.apache.org/jira/browse/FLINK-1476
>             Project: Flink
>          Issue Type: Test
>    Affects Versions: 0.7.0-incubating, 0.8
>         Environment: 3 machines, every machines has 24 CPU cores and allocate 16 CPU
cores for the tests. The memory situation is: 3 * 32G
>            Reporter: xuhong
>            Priority: Critical
>     In the last days, i did some test on flink and spark. The test results shows that
flink can do better on many operations, such as GroupBy, Join and some complex jobs. But when
I do the KMeans, LinearRegression and other loop tests, i found that flink is no more excellent
than spark. I want to konw, whether flink is more comfortable to do the loop jobs with spark.
>     I add code: env.setDegreeOfParallelism(16) in each test to allocate same CPU cores
as in Spark tests.
>     My english is not good, i wish you guys can understand me!
> the following is some config of my Flnk:
> jobmanager.rpc.port: 6123
> jobmanager.heap.mb: 2048
> taskmanager.heap.mb: 2048
> taskmanager.numberOfTaskSlots: 24
> parallelization.degree.default: 72
> jobmanager.web.port: 8081
> webclient.port: 8085
> fs.overwrite-files: true
> taskmanager.memory.fraction: 0.8
> taskmanager.network.numberofBuffers: 70000

This message was sent by Atlassian JIRA

View raw message