lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Making lucene indexing multi threaded
Date Tue, 28 Oct 2014 14:16:09 GMT
bq: When I loop the result set, I reuse the same Document instance.

I really, really, _really_ hope you're calling new for the Document in
the loop. Otherwise that single document will eventually contain all
the data from your entire corpus! I'd expect some other errors to pop
out if you are really doing something like
doc = new Document
for (row in result set) {
  add all the fields
  index the doc
}

but the way you phrased it made me wonder....

BTW, please post the code, it's much easier to see what you're doing that way.

Best,
Erick


On Mon, Oct 27, 2014 at 12:05 PM, Jason Wu <jasonhaowu@gmail.com> wrote:
> Hi Gary,
>
> Thanks for your response. I only call the commit when all my docs are added.
>
> Here is the procedure of my Lucene indexing and re-indexing:
>
>    1. If index data exists inside index directory, remove all the index
>    data.
>    2. Create IndexWriter with 256MB RAMBUFFERSIZE
>    3. Process DB result set
>    - When I loop the result set, I reuse the same Document instance.
>       - At the end of each loop, I call indexWriter.addDocument(doc)
>    4. After all docs are added, call IndexWriter.commit()
>    5. IndexWriter.close();
>
> Thank you,
> Jason
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Making-lucene-indexing-multi-threaded-tp4087830p4166123.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message