mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Frank Scholten (JIRA)" <>
Subject [jira] [Commented] (MAHOUT-1292) lucene2seq creates single document from index
Date Wed, 13 Nov 2013 22:49:21 GMT


Frank Scholten commented on MAHOUT-1292:

Also added checks for the sequential version. CollectionStatistics seems to take only a milisecond
when I ran it on an index with 100000 documents with random strings. Though if anyone knows
a better or more elegant way to check for non-existing fields I am all for it.

> lucene2seq creates single document from index
> ---------------------------------------------
>                 Key: MAHOUT-1292
>                 URL:
>             Project: Mahout
>          Issue Type: Bug
>          Components: Integration
>    Affects Versions: 0.8
>            Reporter: Liz Merkhofer
>            Assignee: Suneel Marthi
>              Labels: cvb, lucene, solr
>             Fix For: 0.9
> Lucene2seq creates only one sequencefile, rather than a file for each document in the
> Running lucene2seq on my Solr (4.3) index produces a file with a header and, it seems,
the field I specified from the index, concatenated for all the documents. After running this
through seq2sparse and rowid (to prepare for cvb), the resulting matrix has only one row,
though it should create one row per document.
> This issue prevents, at least, data from a lucene index from being easily used as input
for cvb. Lucene.vector is also currently inadequate: the keys to its sequence files are LongWriteable,
and rowid will not convert only Text to IntWriteable, as is necessary for the keys in cvb.

This message was sent by Atlassian JIRA

View raw message