mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Grant Ingersoll (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (MAHOUT-165) Using better primitives hash for sparse vector for performance gains
Date Thu, 20 Aug 2009 12:05:15 GMT

    [ https://issues.apache.org/jira/browse/MAHOUT-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745415#action_12745415
] 

Grant Ingersoll edited comment on MAHOUT-165 at 8/20/09 5:05 AM:
-----------------------------------------------------------------

http://acs.lbl.gov/~hoschek/colt-download/releases/license.html 

The clause for military use in HEP is unacceptable, unfortunately.  There is no way we could
enforce that.  If we could use Colt w/o HEP, then we could use Colt.  Perhaps there is just
a class or two from Colt that is ASL licensed that we can copy over and properly attribute
and that doesn't have any dependencies on HEP?  

      was (Author: gsingers):
    http://acs.lbl.gov/~hoschek/colt-download/releases/license.html 

The clause for military use in HEP is unacceptable, unfortunately.  There is no way we could
enforce that.  If we could use Colt w/o HEP, then we could use Colt.  Perhaps there is just
a class or two from Colt that is ASL licensed that we can copy over and properly attribute?
 
  
> Using better primitives hash for sparse vector for performance gains
> --------------------------------------------------------------------
>
>                 Key: MAHOUT-165
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-165
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Matrix
>    Affects Versions: 0.2
>            Reporter: Shashikant Kore
>             Fix For: 0.2
>
>         Attachments: mahout-165.patch
>
>
> In SparseVector, we need primitives hash map for index and values. The present implementation
of this hash map is not as efficient as some of the other implementations in non-Apache projects.

> In an experiment, I found that, for get/set operations, the primitive hash of  Colt performance
an order of magnitude better than OrderedIntDoubleMapping. For iteration it is 2x slower,
though. 
> Using Colt in Sparsevector improved performance of canopy generation. For an experimental
dataset, the current implementation takes 50 minutes. Using Colt, reduces this duration to
19-20 minutes. That's 60% reduction in the delay. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message