mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Frank Scholten (Commented) (JIRA)" <>
Subject [jira] [Commented] (MAHOUT-944) LuceneIndexToSequenceFiles (lucene2seq) utility
Date Sun, 12 Feb 2012 09:50:59 GMT


Frank Scholten commented on MAHOUT-944:

1. Ok. This involves using the FileSystemDirectory from Hadoop contrib, writing a custom InputFormat
and RecordReader which splits the document result across the mappers. Correct?

2. I guess the sort by score would mostly be useful for the sequential version?
> LuceneIndexToSequenceFiles (lucene2seq) utility
> -----------------------------------------------
>                 Key: MAHOUT-944
>                 URL:
>             Project: Mahout
>          Issue Type: New Feature
>          Components: Integration
>    Affects Versions: 0.5
>            Reporter: Frank Scholten
>            Priority: Minor
>             Fix For: 0.7
>         Attachments: MAHOUT-944.patch, MAHOUT-944.patch, MAHOUT-944.patch, MAHOUT-944.patch
> Here is a lucene2seq tool I used in a project. It creates sequence files based on the
stored fields of a lucene index.
> The output from this tool can be then fed into seq2sparse and from there you can do text
> Comes with Java bean configuration.
> Let me know what you think. Some CLI code can be added later on. I used this for a small-scale
project +- 100.000 docs. Is a MR version useful or is that overkill?
> See for commits and review comments
from Simon Willnauer (Thanks Simon!)
> or the attached patch.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message