lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Hatcher <>
Subject Re: Confused again ... Getting at results
Date Sat, 10 Dec 2005 00:17:54 GMT

On Dec 9, 2005, at 3:18 PM, Alan Chandler wrote:
> I am slowly making may way through lucene, as witnessed by earlier  
> threads to
> this mailing list.
> But I am stuck again, going round in circles with the Javadocs.
> I want to display the results of a user entered search where for  
> each document
> I put out a small summary with the searched for words highlighted.
> When I wrote the Analyzer for my documents, I produced the  
> tokenstream  to
> generate Token objects with the start end end positions of each  
> term in them
> Now, from my Hits object I can find each document I need to output,  
> but how do
> I get back to the Tokens I originally produced.

Are you using Lucene 1.4.3?  Or the latest Subversion version?

The Lucene index does not keep all of the information in the Token's  
emitted by the analyzer (unless specified to do so, but 1.4.3 didn't  
support the fancier features).

So, the fail-safe way is to re-tokenize the original text (perhaps  
stored in the Lucene index) and hand that TokenStream to the  

Or you can experiment with the additional Field constructors to  
enable the storage of token offsets and the Highlighter can use those  
for a little better performance, but it's likely to be unnoticeable  
for your application to simply re-tokenize on the fly for only the  
fields you're displaying.  Storing the token offsets increases the  
index size, of course.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message