lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From markharw00d <markharw...@yahoo.co.uk>
Subject Re: Document lazy-loading WAS [Re: Fast access to a random page of the search results.]
Date Tue, 08 Mar 2005 20:16:03 GMT
So this is just the old problem of avoiding reading large, less 
frequently accessed fields when you are trying to read just the smaller 
more frequently accessed fields eg titles.
You can achieve this by:

a)  Modifying Lucene using something like the code I  originally posted 
which stops reading a document's fields if the required list has been 
found (hoping the stored field order is favourable to all of your app's 
access patterns)
b) Storing any large-but-rarely-accessed fields externally eg in files 
or an RDBMS and keeping a primary key in Lucene to retrieve the  content 
when required.
c) If you dont want to use an RDBMS and want to keep it all Lucene-based 
you could store any large-but-rarely-accessed fields in another index 
alongside your main one to avoid the hit.. Each large field could be a 
separate document in this index and you could retrieve it when required 
by accessing largeFieldStoreIndexReader.document(x) where x is
 1) calculated by taking the mainSearchIndexReader document id and 
multiplying by the number of externally stored large fields per main 
document or
 2) Querying largeFieldStoreIndex for a unique key which is stored in 
the mainSearchIndex.

I would favour b) as a and c seem messy.




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message