ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mehdi Seydali <seydali.me...@gmail.com>
Subject Re: deep learning over apache ignite
Date Thu, 20 Dec 2018 05:57:18 GMT
I have another question? As you know dl4j execute over spark. When we want
to integrate dl4j with ignite is it meaningful? For acceleration execution
we can do below idea
1. We can using ignite as cache storage and preparing data for processing
on dl4j.
2. Job on spark can spawn hirarchy job for accelerate execution.
Do you have any comment?

On Wednesday, December 19, 2018, dmitrievanthony <dmitrievanthony@gmail.com>

> Yes, in TensorFlow on Apache Ignite we support distributed learning as you
> described it (please see details in  this documentation
> <https://apacheignite.readme.io/docs/ignite-dataset>  ).
> Speaking about performance, TensorFlow supports distributed learning itself
> (please see details  here
> <https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/
> distribute>
> ). But to start distributed learning in pure TensorFlow you need to setup
> cluster manually, manually distribute training data between cluster nodes
> and handle node failures.
> In TensorFlow on Apache Ignite we do it for you automatically. Apache
> Ignite
> plays cluster manager role, it starts and maintains TensorFlow cluster with
> optimal configuration and handles node failures. At the same time, the
> training is fully performed by TensorFlow anyway. So, the training
> performance is absolutely equal to the case when you use pure TensorFlow
> with proper manually configured and started TensorFlow cluster because we
> don't participate in the training process when the cluster is running
> properly.
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/

View raw message