mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Saikat Kanjilal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAHOUT-1178) GSOC 2013: Improve Lucene support in Mahout
Date Tue, 09 Apr 2013 14:06:17 GMT

    [ https://issues.apache.org/jira/browse/MAHOUT-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13626626#comment-13626626
] 

Saikat Kanjilal commented on MAHOUT-1178:
-----------------------------------------

I had some other questions/thoughts:
1) Ideally I'd think the implementation would belong in the commons or core area since as
you mentioned this would potentially affect multiple algorithms, does that jive with everyone's
thinking on this
2) Can we get away with just storing only the needed parts of the matrix in memory as opposed
to the whole matrix
3) Is it an either or scenario with named vectors versus matrices, pardon my ignorance on
this part since I'm not familiar with named vectors in mahout

Looking forward to begin design/implementation.
                
> GSOC 2013: Improve Lucene support in Mahout
> -------------------------------------------
>
>                 Key: MAHOUT-1178
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1178
>             Project: Mahout
>          Issue Type: New Feature
>            Reporter: Dan Filimon
>              Labels: gsoc2013, mentor
>
> [via Ted Dunning]
> It should be possible to view a Lucene index as a matrix.  This would
> require that we standardize on a way to convert documents to rows.  There
> are many choices, the discussion of which should be deferred to the actual
> work on the project, but there are a few obvious constraints:
> a) it should be possible to get the same result as dumping the term vectors
> for each document each to a line and converting that result using standard
> Mahout methods.
> b) numeric fields ought to work somehow.
> c) if there are multiple text fields that ought to work sensibly as well.
>  Two options include dumping multiple matrices or to convert the fields
> into a single row of a single matrix.
> d) it should be possible to refer back from a row of the matrix to find the
> correct document.  THis might be because we remember the Lucene doc number
> or because a field is named as holding a unique id.
> e) named vectors and matrices should be used if plausible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message