mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dawid Weiss (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAHOUT-253) Proposal for high performance primitive collections.
Date Fri, 24 Sep 2010 12:26:39 GMT

    [ https://issues.apache.org/jira/browse/MAHOUT-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914435#action_12914435
] 

Dawid Weiss commented on MAHOUT-253:
------------------------------------

I'm here, I'm here. Apologies for not following the core development, various reasons behind
it. I also wanted to wait for the HPPC API to stabilize a bit and to get tested in real production
scenarios. I believe this part is over -- we replaced PCJ with HPPC in our code and it works
very well, especially the open internals give a lot of freedom in performance-critical loops.
I realize open internals may become a problem should anything change, but I think weighting
the options it was worth it.

So... I'd love to contribute HPPC to Mahout and proceed with code maintenance as part of Mahout.
The current version of HPPC (0.3.1) is stable. There are 4 JIRA issues in our bug tracking
system, but they are features, not bugs. How do I proceed (and are folks still interested
in integrating this into Mahout)?

> Proposal for high performance primitive collections.
> ----------------------------------------------------
>
>                 Key: MAHOUT-253
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-253
>             Project: Mahout
>          Issue Type: New Feature
>          Components: Utils
>            Reporter: Dawid Weiss
>            Assignee: Dawid Weiss
>            Priority: Minor
>         Attachments: hppc-1.0-dev.zip
>
>
> A proposal for template-driven collections library (lists, sets, maps, deques), with
specializations for Java primitive types to save memory and increase performance. The "templates"
are regular Java classes written with generics and certain "intrinsics", that is blocks replaceable
by a regexp-preprocessor. This lets one write the code once, immediately test it (tests are
also templates) and generate primitive versions from a single source.
> An additional interesting part is the benchmarking subsystem written on top of JUnit
;)
> There are major differences from the Java Collections API, most notably no interfaces
and interface-compatible views over sub-collections or key/value sets. These classes also
expose their internal implementation (buffers, addressing, etc.) so that the code can be optimized
for a particular use case.
> These motivations are further discussed here, together with an API overview.
> http://www.carrot-search.com/download/hppc/index.html
> I am curious what you think about it. If folks like it, Carrot Search will donate the
code to Mahout (or Apache Commons-?) and will maintain it (because we plan to use it in our
internal projects anyway).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message