mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: [Slightly OT] SVD etc on Map/Reduce?
Date Fri, 15 Jan 2010 06:26:17 GMT
On Thu, Jan 14, 2010 at 10:09 PM, Philipp K. Janert <janert@ieee.org> wrote:

> > If you mean matrix factorization, take a look at this:
> > http://arxiv.org/abs/0909.4061v1
>
> That seems to support my earlier hunch that
> efficient implementations of such factorizations
> on M/R would likely be approximate only or
> partial (ie yielding the largest of the eigenvalues,
> not necessarily the entire spectrum).
>

For very large sparse problems, approximate decompositions are generally
preferred.  Due to limited accuracy in the input, only the first several
eigenvectors can be extracted at all.  Moreover, many important problems
have very large apparent dimensionality, but limited actual rank.   Neither
of these characteristics is a characteristic of map-reduce in the slightest.


 > It is the common sense of those on this mailing list that these kinds of
> > algorithms could be done using map-reduce.
>
> I am not sure what you are trying to tell me here.


I am trying to say that we don't yet have working implementations.  There
should be a k-means implementation that use these techniques before long.
You would be very welcome to try your hand at some other of the algorithms
and I am sure that you would have quite a lot of support from the mailing
list.

You comments puzzle me, though.  Do you have an application in mind?  Was
there something you were particularly looking for?

-- 
Ted Dunning, CTO
DeepDyve

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message