lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson" <erickerick...@gmail.com>
Subject Re: Index Rows as Documents? Help me design a solution
Date Tue, 25 Jul 2006 18:39:12 GMT
The code looks good, *assuming* that the IndexWriter you pass in isn't
closed/opened between files (this would  be a problem if you have lots of
files to index......). I've had the IndexWriter.optimize method take a
loooong time to complete, so I typically don't do this until I'm entirely
done...

My application is indexing 10,000 documents in 15 seconds or so. These are
XML documents that need to be parsed and a dozen or so fields indexed FWIW.

An easy test for narrowing down where your problem is would be to just
comment out the writer.addDocument and take some timings. Then, perhaps,
comment out the open/close/optimize of the IndexWriter and see if *that*
made a measurable difference.

But if you're not opening and closing the index writer between files, I'm
stumped. I usually take a "divide and conquer" approach or haul out a
performance analyzer :(.

My experience is that Lucene indexes quite quickly, so I would assume it was
a problem in my code for quite a while before throwing in the towel.

You aren't by chance reading/writing over a network that may be slow are you
? (really grasping at straws here.....).

Best
Erick

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message