spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roope Astala <roast...@microsoft.com.INVALID>
Subject RE: Do we anything for Deep Learning in Spark?
Date Thu, 06 Jul 2017 22:29:27 GMT
You can use an attached GPU VM for DNN training, and do other processing on regular CPU nodes.
You can even deallocate the GPU VM to save costs when not using it. The GPU branch has instructions
how to set up such compute environment:

https://github.com/Azure/mmlspark/tree/gpu#gpu-vm-setup

Cheers,
Roope – Microsoft Cloud AI Team

From: hosur narahari [mailto:hnr1992@gmail.com]
Sent: Thursday, July 6, 2017 1:54 AM
To: Gaurav1809 <gauravhpandya@gmail.com>
Cc: user <user@spark.apache.org>
Subject: Re: Do we anything for Deep Learning in Spark?

Hi Roope,

Does this mmlspark project uses GPGPU for processing and just CPU cores since DL models are
computationally very intensive.

Best Regards,
Hari

On 6 Jul 2017 9:33 a.m., "Gaurav1809" <gauravhpandya@gmail.com<mailto:gauravhpandya@gmail.com>>
wrote:
Thanks Roope for the inputs.

On Wed, Jul 5, 2017 at 11:41 PM, Roope [via Apache Spark User List] <[hidden email]<http://user/SendEmail.jtp?type=node&node=28826&i=0>>
wrote:
Microsoft Machine Learning Library for Apache Spark lets you run CNTK deep learning models
on Spark.

https://github.com/Azure/mmlspark<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fmmlspark&data=02%7C01%7Croastala%40microsoft.com%7Cc8e41a789e9d4837c81408d4c4337609%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636349172685703933&sdata=1FqnrRsLY6JrPb%2BdkG4AlLBN%2F8f2O6w3Ehi9DYvu6lc%3D&reserved=0>

The library APIs are focused on image processing scenarios, and are compatible with SparkML
Pipelines.

Cheers,
Roope - Microsoft Cloud AI Team
________________________________
If you reply to this email, your message will be added to the discussion below:
http://apache-spark-user-list.1001560.n3.nabble.com/Do-we-anything-for-Deep-Learning-in-Spark-tp28772p28824.html<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-spark-user-list.1001560.n3.nabble.com%2FDo-we-anything-for-Deep-Learning-in-Spark-tp28772p28824.html&data=02%7C01%7Croastala%40microsoft.com%7Cc8e41a789e9d4837c81408d4c4337609%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636349172685703933&sdata=zFajYYH16xwzDRwgP1se9m%2BAA5fIufUS7FFxHjzTOl0%3D&reserved=0>
To start a new topic under Apache Spark User List, email [hidden email]<http://user/SendEmail.jtp?type=node&node=28826&i=1>
To unsubscribe from Do we anything for Deep Learning in Spark?, click here.
NAML<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-spark-user-list.1001560.n3.nabble.com%2Ftemplate%2FNamlServlet.jtp%3Fmacro%3Dmacro_viewer%26id%3Dinstant_html%2521nabble%253Aemail.naml%26base%3Dnabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace%26breadcrumbs%3Dnotify_subscribers%2521nabble%253Aemail.naml-instant_emails%2521nabble%253Aemail.naml-send_instant_email%2521nabble%253Aemail.naml&data=02%7C01%7Croastala%40microsoft.com%7Cc8e41a789e9d4837c81408d4c4337609%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636349172685703933&sdata=wzDT2AoJRfwqxlumkdoRlGq6b0PaeoypZtvvLgFlR1M%3D&reserved=0>


________________________________
View this message in context: Re: Do we anything for Deep Learning in Spark?<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-spark-user-list.1001560.n3.nabble.com%2FDo-we-anything-for-Deep-Learning-in-Spark-tp28772p28826.html&data=02%7C01%7Croastala%40microsoft.com%7Cc8e41a789e9d4837c81408d4c4337609%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636349172685703933&sdata=dDprNB1TSm9WkAY1iuN1NjkG7vzx6JaiXB42DcXqUt4%3D&reserved=0>
Sent from the Apache Spark User List mailing list archive<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-spark-user-list.1001560.n3.nabble.com%2F&data=02%7C01%7Croastala%40microsoft.com%7Cc8e41a789e9d4837c81408d4c4337609%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636349172685703933&sdata=bI5METH7hpg6xCjUQF0%2BiIKY7qFy709YGC4g0n9MjW0%3D&reserved=0>
at Nabble.com.
Mime
View raw message