Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Subject: Displaying search result data - stored fields vs external source
From: Joel Halbert <joel@su3analytics.com>
To: Lucene Users <java-user@lucene.apache.org>
Content-Type: text/plain
Organization: SU3 Analytics
Date: Tue, 15 Sep 2009 09:19:50 +0100
Message-Id: <1253002790.7551.20.camel@bohr>
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit

Hi,

When using Lucene I always consider two approaches to displaying search
result data to users:

1. Store any fields that we index and display to users in the Lucene
Documents themselves. When we perform a search simply retrieve the data
to be displayed from the Lucence documents themselves.

or

2. Index fields in Lucene but reference data to be displayed from
another source, such as a database. So, when searching I would search
for documents then use a (stored) reference key on the documents to then
lookup the display fields to display from another source e.g. a
database.

With regards to the number and size of stored fields I am looking at
indexing and displaying approximately 4 relatively small fields for each
document (e.g.  name, age, short description, URL ~ approx 500bytes in
total). In any query about 10 hits will be displayed to the user.
Approximately 10 million documents to index and search.

I am interested the differences in both approaches with regards to:

1) Indexing time performance (how long it might take to index with and
without stored fields)
2) Search time performance (total time taken to search for matching
documents and then display fields to users)

I am less interested in differences arising from
maintainability/increased storage requirements.

I would be interested to see what others  think of using each approach.

Cheers,
Joel


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org