lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anthony Urso <anthony.u...@gmail.com>
Subject Re: Field with reader limitation arbitrary
Date Tue, 15 Sep 2009 03:05:53 GMT
It's best to file a feature request on the Lucene issue tracker if you
are interested in seeing this implemented.

http://issues.apache.org/jira/browse/LUCENE

Just cut and paste your description and attach a patch and/or tests if
you have them.

Cheers,
Anthony

On Mon, Sep 14, 2009 at 1:03 PM, Glen Newton <glen.newton@gmail.com> wrote:
> Hi,
>
> In 2.4.1, Field has 2 constructors that involve a Reader:
> public Field(String name,
>                  Reader reader)
> public Field(String name,
>                  Reader reader,
>                  Field.TermVector termVector)
>
> http://lucene.apache.org/java/2_4_1/api/core/org/apache/lucene/document/Field.html#Field(java.lang.String,%20java.io.Reader,%20org.apache.lucene.document.Field.TermVector)
>
> The Reader references a text file on the filesystem. These
> constructors do the following:
> "Create a tokenized and indexed field that is not stored, optionally
> with storing term vectors. The Reader is read only when the Document
> is added to the index, i.e. you may not close the Reader until
> IndexWriter.addDocument(Document)  has been called."
>
> Someone has made the decision that we will not be interested in
> storing files read using a Reader (at least not with these
> constructors).
> This is rather arbitrary.
> As someone who has massively parralelized my indexing AND sometimes
> might want to also store files in the index,  having a queue of 1000
> Documents with 1000 Readers to files is vastly preferable to having
> 1000 documents with 1000 (perhaps very large) Strings with all the
> contents of the files. While this is not the best for all cases (#open
> file handles, etc), this is a use case which would benefit from being
> able to do this (i.e. reduced memory footprint, especially for large
> files or large queues).
>
> Suggestion: replace or add a constructor with:
> public Field(String name,
>             Reader reader,
>             Field.Store store,
>             Field.Index index,
>             Field.TermVector termVector)
>
> Constructively,
>
> Glen Newton
>  http://zzzoot.blogspot.com/
>  http://lab.cisti-icist.nrc-cnrc.gc.ca/cistilabswiki/index.php/LuSql
>
> -
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message