Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01CB5E38.1D9D85D0"
Subject: Questions about Lucene usage recommendations
Date: Mon, 27 Sep 2010 13:35:20 +0200
Message-ID: 
 <F1AD6F6673633C4692F7689E42FF9AAA42138C@ADAPA.ge-admin.ad.etat-ge.ch>
Thread-Topic: Questions about Lucene usage recommendations
Thread-Index: ActeOBEXIUoENNJLTmWJxuPcrTUc4w==
From: "Pawlak Michel (DCTI)" <michel.pawlak@etat.ge.ch>
To: <java-user@lucene.apache.org>

------_=_NextPart_001_01CB5E38.1D9D85D0
Content-Type: text/plain;
	charset="US-ASCII"
Content-Transfer-Encoding: quoted-printable

Hello,

We have an application which is using lucene and we have strong
performance issues (on bad days, some searches take more than 2
minutes). I'm new to the Lucene component, thus I'm not sure Lucene is
correctly used and thus would like to have some information on lucene
usage recommendations. This would help locate the problem (application
code / lucene configuration / hardware / all) It would be great if a
project committer / specialist could answer those questions.

First some facts about the application :=20
- Lucene version being used : 2.1.0 (february 2007...)
- around 1.4M "documents" to be indexed.
- Db size (all data to be indexed is stored in DB fields) : 3.5 GB
- Index file size on disk : 1.6 GB (note that one cfs file is 780M,
another one is 600M, the rest consists of smaller files)
- single indexer, multiple readers (6 readers)
- around 150 documents are modified per day
- indexing is done right after every modification
- simple searches can take ages (for instance searching for "chocolate"
could take for more than 2 minutes)
- I do not have access to source code (yes that's the funny part)

My questions :=20
- Is this version of Lucene still supported ?
- What are the main reasons, if any, one should use the latest version
of lucene instead of 2.1.0 ? (for instance : performance, stability,
critical fixes, support, etc.) (the answer may sound obvious, but I
would like to have an official answer)
- Is there any recommendation concerning storage any Lucene user should
know (not benchmarks, but recommendations such as "better use physical
HDD", "do not use NFS if possible", "if your cfs files are greater than
XYZ, better use this kind of storage", "if you have more than XYZ
searches per second, better..." etc)
- Is there any recommandation concerning cfs file size ?=20
- Is there a way to limit the size of cfs files ?=20
- What is the impact on search performance if cfs file size is limited ?
- How often should optimization occur ? (every day, week, month ?)
- I saw that IndexWriter has methods such as setMaxFieldLength()
setMergeFactor() setMaxBufferedDocs() setMaxMergeDocs() Can you briefly
explain how these can affect performance ?
- Is there any other recommandation "dummies" should be informed of, and
every expert has to know ? For instance as a list of lucene patterns /
anti patterns which may affect performance.

If my questions are not precise enough, do not hesitate to ask for
details. If you see an obvious problem do not hesitate to tell me.

A big thank you in advance for your help,

Best regards,

Michel


------_=_NextPart_001_01CB5E38.1D9D85D0--