spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zeweichen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-16357) After enabling Spark shuffle RPC encryption using 3DES, Sparksql query has poor performance when running in parallel.
Date Mon, 04 Jul 2016 05:24:10 GMT

     [ https://issues.apache.org/jira/browse/SPARK-16357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

zeweichen updated SPARK-16357:
------------------------------
    Fix Version/s:     (was: 2.1.0)

> After enabling Spark shuffle RPC encryption using 3DES, Sparksql query has poor performance
when running in parallel.
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-16357
>                 URL: https://issues.apache.org/jira/browse/SPARK-16357
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.6.1
>         Environment: Apache Spark 1.6.1 
> BigBench 1.0.1
>            Reporter: zeweichen
>            Priority: Minor
>
> During our performance test of Spark shuffle RPC encryption using 3DES on Sparksql, we
found that throughput test of 2 stream lasts 2.68X time than power test, which is much larger
than it should be. Q30 cost 0.5 hour in stream1,  but q30 last 2.5 hours in stream 0 which
is much longer than it should be.  This caused that queries in stream1 go ahead much faster
than stream0. So it works like single stream and can not fully use CPU resource.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message