incubator-cvs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Incubator Wiki] Trivial Update of "HornProposal" by edwardyoon
Date Wed, 26 Aug 2015 12:43:20 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Incubator Wiki" for change notification.

The "HornProposal" page has been changed by edwardyoon:
https://wiki.apache.org/incubator/HornProposal?action=diff&rev1=9&rev2=10

Comment:
minor changes

  
  == Rationale ==
  
- While many of deep learning open source softwares such as Caffe, DeepDist, and NeuralGiraph
are still data or model parallel only, we aim to support both data and model parallelism and
also fault-tolerant system design. The basic idea of data and model parallelism is use of
the remote parameter server to parallelize model creation and distribute training across machines,
and the BSP framework of Apache Hama for performing asynchronous mini-batches. Within single
BSP job, each task group works asynchronously using region barrier synchronization instead
of global barrier synchronization, and trains large-scale neural network model using assigned
data sets in BSP paradigm. Thus, we achieve data and model parallelism. This architecture
is inspired by Google's !DistBelief (Jeff Dean et al, 2012).
+ While many of deep learning open source softwares such as Caffe, DeepDist, DN4j, and NeuralGiraph
are still data or model parallel only, we aim to support both data and model parallelism and
also fault-tolerant system design. The basic idea of data and model parallelism is use of
the remote parameter server to parallelize model creation and distribute training across machines,
and the BSP framework of Apache Hama for performing asynchronous mini-batches. Within single
BSP job, each task group works asynchronously using region barrier synchronization instead
of global barrier synchronization, and trains large-scale neural network model using assigned
data sets in BSP paradigm. Thus, we achieve data and model parallelism. This architecture
is inspired by Google's !DistBelief (Jeff Dean et al, 2012).
  
  == Initial Goals ==
  

---------------------------------------------------------------------
To unsubscribe, e-mail: cvs-unsubscribe@incubator.apache.org
For additional commands, e-mail: cvs-help@incubator.apache.org


Mime
View raw message