spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ulanov, Alexander" <>
Subject RE: Support off-loading computations to a GPU
Date Mon, 04 Jan 2016 21:11:37 GMT
Hi Kazuaki,

Sounds very interesting! Could you elaborate on your benchmark with regards to logistic regression
(LR)? Did you compare your implementation with the current implementation of LR in Spark?

Best regards, Alexander

From: Kazuaki Ishizaki []
Sent: Sunday, January 03, 2016 7:52 PM
Subject: Support off-loading computations to a GPU

Dear all,

We reopened the existing JIRA entry support
off-loading computations to a GPU by adding a description for our prototype. We are working
to effectively and easily exploit GPUs on Spark at Please
also visit our project page

For now, we added a new format for a partition in an RDD, which is a column-based structure
in an array format, in addition to the current Iterator[T] format with Seq[T]. This reduces
data serialization/deserialization and copy overhead between CPU and GPU.

Our prototype achieved more than 3x performance improvement for a simple logistic regression
program using a NVIDIA K40 card.

This JIRA entry (SPARK-3785) includes a link to a design document. We are very glad to hear
valuable feedback/suggestions/comments and to have great discussions to exploit GPUs in Spark.

Best Regards,
Kazuaki Ishizaki

View raw message