lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saurabh Gokhale <saurabhgokh...@gmail.com>
Subject Regarding multiple index creation and Searching
Date Mon, 15 Aug 2011 17:14:51 GMT
Hi All,

In my application, we have been maintaining lucene index for the 3 years
worth of data. (more than 70GB of single lucene index). To improve
performance, recently it was decided to break indexes into 1 year worth of
data each (3 indexes). before we work on the required change, I wanted to
get clarification on few questionsMy setup has following properties

OS: Red Hat Enterprise
Lucene: 3.2
Java: JDK 1.6

Questions:
1. What is the average acceptable size for Lucene index that is considered
OK for searching? (before it is broken down into multiple indexes)
2. Other than performance, what should be the criteria to decide on
separating the index into mutiple index. (Criteria like single file in the
index should not be more than 2GB, or the total lucene index folder size
should not be above 10GB etc)


(Regarding code changes required to break the documents into appropriate
year)
I will be reindexing all the documents again using modified code base. For
that I will be required to

3. Create multiple indexWriters and index the document using appropriate
writer as per the date of the document.
4. While searching, use multiSearcher or ParallelMultiSearcher to search
across all indexes at once.

do I have to make any other change as far as the code changes are concerned?

Thanks

Saurabh

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message