mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: [GSOC] Matrix Operations on HDFS
Date Sun, 30 May 2010 18:52:54 GMT
The idea with those deprecations is that we imported a whole bunch of code
from the Colt package but it didn't have enough (any) unit tests.  To avoid
institutionalizing bugs, we marked everything in that import as deprecated
and are removing the deprecations as we need to use those methods and as we
test them.

If you need those deprecated methods, then go right ahead and write some
unit tests and undeprecate them.  While you are at it, make sure that you
address the checkstyle warnings.  If you have questions about how to do
this, feel free to ask about specific files.

None of these deprecated methods are likely to be distributed although they
may be of considerable use in a distributed implementation.

Jake's has implemented a kind of a distributed matrix that is useful for
certain kinds of multiplications.  Can you say more about what you need?  It
is possible that a small variant of what he has would work for you.

On Sun, May 30, 2010 at 4:51 AM, Sisir Koppaka <sisir.koppaka@gmail.com>wrote:

> Hi,
> I was looking for distributed map-reduce based 1D, 2D, and 3D operations on
> HDFS for the RBM algorithm. o.a.m.math.matrix has them but they are marked
> "@deprecated until unit tests are in place.  Until this time, this
> class/interface is unsupported."
>
> Jake posted about  o.a.m.math.hadoop.decomposer.DistributedLanczosSolver in
> Shannon's thread a few days ago - is there something like that for
> distributed map-reduce operations on HDFC for generic matrices? I need
> these
> operations because they don't fit in memory for large datasets.
>
> --
> Sisir
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message