spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ulanov, Alexander" <alexander.ula...@hp.com>
Subject RE: Using CUDA within Spark / boosting linear algebra
Date Wed, 01 Apr 2015 19:11:03 GMT
FYI, I've added instructions to Netlib-java wiki, Sam added the link to them from the project's
readme.md
https://github.com/fommil/netlib-java/wiki/NVBLAS

Best regards, Alexander
-----Original Message-----
From: Xiangrui Meng [mailto:mengxr@gmail.com] 
Sent: Monday, March 30, 2015 2:43 PM
To: Sean Owen
Cc: Evan R. Sparks; Sam Halliday; dev@spark.apache.org; Ulanov, Alexander; jfcanny
Subject: Re: Using CUDA within Spark / boosting linear algebra

Hi Alex,

Since it is non-trivial to make nvblas work with netlib-java, it would be great if you can
send the instructions to netlib-java as part of the README. Hopefully we don't need to modify
netlib-java code to use nvblas.

Best,
Xiangrui

On Thu, Mar 26, 2015 at 9:54 AM, Sean Owen <sowen@cloudera.com> wrote:
> The license issue is with libgfortran, rather than OpenBLAS.
>
> (FWIW I am going through the motions to get OpenBLAS set up by default 
> on CDH in the near future, and the hard part is just handling
> libgfortran.)
>
> On Thu, Mar 26, 2015 at 4:07 PM, Evan R. Sparks <evan.sparks@gmail.com> wrote:
>> Alright Sam - you are the expert here. If the GPL issues are 
>> unavoidable, that's fine - what is the exact bit of code that is GPL?
>>
>> The suggestion to use OpenBLAS is not to say it's the best option, 
>> but that it's a *free, reasonable default* for many users - keep in 
>> mind the most common deployment for Spark/MLlib is on 64-bit linux on EC2[1].
>> Additionally, for many of the problems we're targeting, this 
>> reasonable default can provide a 1-2 orders of magnitude improvement 
>> in performance over the f2jblas implementation that netlib-java falls back on.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org For 
> additional commands, e-mail: dev-help@spark.apache.org
>
Mime
View raw message