spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Ash <and...@andrewash.com>
Subject Re: Scala vs Python performance differences
Date Wed, 12 Nov 2014 10:12:37 GMT
Jeremy,

Did you complete this benchmark in a way that's shareable with those
interested here?

Andrew

On Tue, Apr 15, 2014 at 2:50 PM, Nicholas Chammas <
nicholas.chammas@gmail.com> wrote:

> I'd also be interested in seeing such a benchmark.
>
>
> On Tue, Apr 15, 2014 at 9:25 AM, Ian Ferreira <ianferreira@hotmail.com>
> wrote:
>
>> This would be super useful. Thanks.
>>
>> On 4/15/14, 1:30 AM, "Jeremy Freeman" <freeman.jeremy@gmail.com> wrote:
>>
>> >Hi Andrew,
>> >
>> >I'm putting together some benchmarks for PySpark vs Scala. I'm focusing
>> on
>> >ML algorithms, as I'm particularly curious about the relative performance
>> >of
>> >MLlib in Scala vs the Python MLlib API vs pure Python implementations.
>> >
>> >Will share real results as soon as I have them, but roughly, in our
>> hands,
>> >that 40% number is ballpark correct, at least for some basic operations
>> >(e.g
>> >textFile, count, reduce).
>> >
>> >-- Jeremy
>> >
>> >---------------------
>> >Jeremy Freeman, PhD
>> >Neuroscientist
>> >@thefreemanlab
>> >
>> >
>> >
>> >--
>> >View this message in context:
>> >
>> http://apache-spark-user-list.1001560.n3.nabble.com/Scala-vs-Python-perfor
>> >mance-differences-tp4247p4261.html
>> >Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>>
>>
>

Mime
View raw message