hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@gmail.com>
Subject Re: Multilayer Perceptron
Date Thu, 15 Nov 2012 09:34:23 GMT
Hi Christian,

I followed your work at Mahout from the beginning of the year, I actually
wanted to introduce BSP to you back then, but I lost track.
Yes it is suitable for the kind of algorithm you proposed.

However I came across a talk of Jeff Dean who told about the deep learning
framework at Google, I filed a jira with more details [1]. It depends on a
more lightweight synchronization behaviour, since global synchronizations
are very costly for that kind of computation paradigm.

Your approach will make use of the normal barrier synchronization, so it
would be cool to get something equally running. Although a deep learning
framework would superseed your work.

Just curious, for what kind of stuff do you need colt? The problem is
majorly that this is another dependency that isn't documented and tested
well, so we mostly disagree in adding it to our trunk. Maybe we can
reimplement the things that you need colt for because we have our own math
lib in the ML module.

Regards,
Thomas


[1] https://issues.apache.org/jira/browse/HAMA-675

2012/11/15 info@christianherta.de <info@christianherta.de>

> Dear All,
> what do you think about to scale out the learning of Multi Layer
> Perceptrons
> (MLP) with BSPs?
> I heard the talk of Tommaso at the apacheConAt first glance the pramming
> model
> BSP seems to fit better the MapReduce for this purpose.
> The basic idea is to distribute the backprop algorithm is the following:
>
> Distribution of learning can be done by (batch learning):
> 1 Partioning of the data in x chunks
> 2 On each working node: Learning the weight changes (as matrices) in each
> chunk
> 3 Combining the matrixes (weight changes) and simultaneous update of the
> weights
> in each node - back to 2
>
> Maybe this procedure can be done with random parts of the chunks
> (distributed
> quasi online learning).
>
> I wrote the (basic) backprob algorithm of a multi layer preceptron (see
> mahout
> patch https://issues.apache.org/jira/browse/MAHOUT-976). It uses the
> Mahout
> Matrix Library, which is under the hood the Colt Matrix Library from Cern.
>
> Probably using the Cern Matrix Library would also suitable for Hama. Then
> it
> could be easy to port the MLP to Hama.
>
> What do you think about it?
>
> Thanks for you response.
>
> Cheers
>  Christian

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message