mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jake Mannix (JIRA)" <>
Subject [jira] Updated: (MAHOUT-165) Using better primitives hash for sparse vector for performance gains
Date Wed, 18 Nov 2009 00:45:39 GMT


Jake Mannix updated MAHOUT-165:

    Attachment: MAHOUT-165-with-colt.patch

While there isn't yet a consensus on where this stuff will live, for now, so we can see what
we're getting into, I've attached a patch which includes, currently in core/src/main/java,
two new packages: org.apache.mahout.colt and org.apache.mahout.jet.

The only modifications I've done to these files is removing any dependency on hep.aida.*,
the LGPL'ed corejava.*, and cern.clhep physical constants stuff, and I also, ironically enough,
removed an ugly "©" symbol which at least on my system was causing javac to blow up during
compile (I left the rest of the entire copyright notice intact, just without the © symbol
in there). 

I added:

 * @deprecated until unit tests are in place.  Until this time, this class/interface is unsupported.

in front of every public class or interface.

Also, to get it to compile, I added a maven dependency on Doug Lea's concurrent jar, which
it turns out is accessible in apache's maven repo.

If we really want this out in google-code somewhere, I can easily package up this patch and
put it somewhere else.

> Using better primitives hash for sparse vector for performance gains
> --------------------------------------------------------------------
>                 Key: MAHOUT-165
>                 URL:
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Matrix
>    Affects Versions: 0.2
>            Reporter: Shashikant Kore
>            Assignee: Grant Ingersoll
>             Fix For: 0.3
>         Attachments: colt.jar, mahout-165-trove.patch, MAHOUT-165-updated.patch, MAHOUT-165-with-colt.patch,
mahout-165.patch, MAHOUT-165.patch, mahout-165.patch
> In SparseVector, we need primitives hash map for index and values. The present implementation
of this hash map is not as efficient as some of the other implementations in non-Apache projects.

> In an experiment, I found that, for get/set operations, the primitive hash of  Colt performance
an order of magnitude better than OrderedIntDoubleMapping. For iteration it is 2x slower,
> Using Colt in Sparsevector improved performance of canopy generation. For an experimental
dataset, the current implementation takes 50 minutes. Using Colt, reduces this duration to
19-20 minutes. That's 60% reduction in the delay. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message