mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shashikant Kore (JIRA)" <>
Subject [jira] Updated: (MAHOUT-165) Using better primitives hash for sparse vector for performance gains
Date Wed, 18 Nov 2009 13:50:39 GMT


Shashikant Kore updated MAHOUT-165:

    Attachment: mahout-165-18nov-updated.patch

I am updating the patch to ensure hashCode() is evaluated correctly. Since hashCode() needs
elements in sorted order and many other routines are OK with any order, I have added an extra
method to return iterator with sorted elements. Only SparseVector handles the sorting flag.

This is built on Drew's patch. So, the namespaces are changed from colt to  org.apache.mahout.colt.*

> Using better primitives hash for sparse vector for performance gains
> --------------------------------------------------------------------
>                 Key: MAHOUT-165
>                 URL:
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Matrix
>    Affects Versions: 0.2
>            Reporter: Shashikant Kore
>            Assignee: Grant Ingersoll
>             Fix For: 0.3
>         Attachments: colt.jar, mahout-165-18nov-updated.patch, mahout-165-18nov.patch,
mahout-165-trove.patch, MAHOUT-165-updated.patch, MAHOUT-165-with-colt-module.patch, MAHOUT-165-with-colt.patch,
mahout-165.patch, MAHOUT-165.patch, mahout-165.patch
> In SparseVector, we need primitives hash map for index and values. The present implementation
of this hash map is not as efficient as some of the other implementations in non-Apache projects.

> In an experiment, I found that, for get/set operations, the primitive hash of  Colt performance
an order of magnitude better than OrderedIntDoubleMapping. For iteration it is 2x slower,
> Using Colt in Sparsevector improved performance of canopy generation. For an experimental
dataset, the current implementation takes 50 minutes. Using Colt, reduces this duration to
19-20 minutes. That's 60% reduction in the delay. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message