spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Huang, Jie" <jie.hu...@intel.com>
Subject [SparkScore]Performance portal for Apache Spark - WW30
Date Mon, 27 Jul 2015 03:10:27 GMT
Performance Portal for Apache Spark
Description
________________________________
Each data point represents each workload runtime percent compared with the previous week.
Different lines represents different workloads running on spark yarn-client mode.
Hardware
________________________________
CPU type: Intel(r) Xeon(r) CPU E5-2697 v2 @ 2.70GHz
Memory: 128GB
NIC: 10GbE
Disk(s): 8 x 1TB SATA HDD
Software
________________________________
JAVA version: 1.8.0_25
Hadoop version: 2.5.0-CDH5.3.2
HiBench version: 4.0
Spark on yarn-client mode
Cluster
________________________________
1 node for Master
10 nodes for Slave
Regular
Summary
The lower percent the better performance.
________________________________
Group

ww24

ww25

ww26

ww27

ww28

ww29

ww30

HiBench

-6.5%

-3.1%

-2.1%

-6.4%

-2.7%

-0.7%

2.2%

spark-perf

-4.7%

-4.6%

-5.4%

-4.6%

-12.8%

-12.5%

-12.3%

[http://01org.github.io/sparkscore/image/plaf1.time/overall.png]
Y-Axis: normalized completion time; X-Axis: Work Week.
The commit number can be found in the result table.
The performance score for each workload is normalized based on the elapsed time for 1.2 release.
The lower the better.
Detail
________________________________
HiBench
________________________________
JOB

ww24

ww25

ww26

ww27

ww28

ww29

ww30

commit

db81b9d8

4eb48ed1

32e3cdaa

ec784381

2b820f2a

c472eb17

a803ac3e

sleep

-4.1%

12.8%

-5.1%

-4.5%

-3.1%

-0.7%

-0.6%

wordcount

-18.6%

-10.9%

6.9%

-12.9%

-10.0%

-9.2%

-8.0%

kmeans

86.9%

95.8%

123.3%

99.3%

127.9%

102.6%

95.3%

scan

-25.5%

-21.0%

-12.4%

-19.8%

-19.7%

-20.5%

8.6%

bayes

-29.7%

-31.3%

-30.9%

-31.1%

-31.0%

-30.1%

-27.6%

aggregation

-15.3%

-15.0%

-37.6%

-37.0%

-37.3%

7.6%

12.9%

join

-12.7%

-13.9%

-16.4%

-17.8%

-14.8%

-13.2%

-15.6%

sort

-17.5%

-17.3%

-20.7%

-17.7%

-13.9%

-15.6%

-16.9%

pagerank

-11.4%

-13.0%

-11.4%

-10.1%

-12.0%

-11.7%

-11.0%

terasort

-16.7%

-17.0%

-16.3%

-11.9%

-13.1%

-15.7%

-14.5%

Comments: null means no such workload running or workload failed in this time.
[http://01org.github.io/sparkscore/image/plaf1.time/HiBench_workloads.png]
Y-Axis: normalized completion time; X-Axis: Work Week.
The commit number can be found in the result table.
The performance score for each workload is normalized based on the elapsed time for 1.2 release.
The lower the better.
spark-perf
________________________________
JOB

ww24

ww25

ww26

ww27

ww28

ww29

ww30

commit

db81b9d8

4eb48ed1

32e3cdaa

ec784381

2b820f2a

c472eb17

a803ac3e

agg

5.2%

2.5%

1.1%

3.0%

-18.8%

-19.0%

-18.4%

agg-int

4.0%

8.2%

7.0%

7.5%

6.2%

11.4%

5.5%

agg-naive

-6.7%

-6.8%

-8.5%

-6.9%

-15.5%

-18.0%

-20.6%

scheduling

-6.4%

-6.5%

-5.7%

-1.8%

-6.0%

-9.7%

-1.0%

count-filter

-10.2%

-10.4%

-9.8%

-10.4%

-18.0%

-17.4%

-19.6%

count

-7.3%

-7.0%

-8.0%

-7.4%

-15.1%

-14.3%

-16.5%

sort

-14.6%

-14.4%

-13.9%

-15.9%

-24.0%

-23.2%

-21.7%

sort-int

-1.5%

-2.2%

-5.3%

-5.0%

-11.3%

-9.6%

-6.4%

Comments: null means no such workload running or workload failed in this time.
[http://01org.github.io/sparkscore/image/plaf1.time/spark-perf_workloads.png]
Y-Axis: normalized completion time; X-Axis: Work Week.
The commit number can be found in the result table.
The performance score for each workload is normalized based on the elapsed time for 1.2 release.
The lower the better.

Mime
View raw message