lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sirish Vadala <>
Subject Re: Problem searching in the same sentence
Date Thu, 30 Sep 2010 00:20:47 GMT

Hello All:

I am performing the sentence specific phrase search, by adding sentence by
sentence to the same field as suggested below. Everything works fine, but
when I display my results, highlighter is not able to find the search text

The following is my code:

SentenceScanner sentenceScanner = new
SentenceScanner(doc.getText().replaceAll("\\s+", " "));
ArrayList<String> sentencesList = sentenceScanner.getAllSentences();
for (String sentence : sentencesList){
	addFieldToDocument(document, IFIELD_TEXT, sentence, true, true);

private void addFieldToDocument(Document document, String fieldName,
	String fieldValue, Boolean store, Boolean tokenize) {
	String validFieldValue = Utility.validateString(fieldValue);
	Field field = new Field(fieldName, validFieldValue,
			(store) ? Field.Store.YES : Field.Store.NO,
			(tokenize) ? Field.Index.ANALYZED : Field.Index.NOT_ANALYZED);

My custom standard analyzer:

public class MyStandardAnalyzer extends StandardAnalyzer implements
IndexFields {
	public MyStandardAnalyzer(Version matchVersion) {
	public int getPositionIncrementGap(String fieldName) {
		int incrementGap = super.getPositionIncrementGap(fieldName);
		if (fieldName.equals(IFIELD_TEXT)) {
			incrementGap += 10;
		return incrementGap;

My highlighter code:

//analyzer instantiated as 'MyStandardAnalyzer' in the constructor
public String highlight(String text) {
   String highlightedText = "";
   TokenStream tokenStream = analyzer.tokenStream(IndexFields.IFIELD_TEXT,
new StringReader(text));
   try {
	return highlighter.getBestFragments(tokenStream, text,
			maxFragments, delimiter);
   } catch (Exception e) {
	return highlightedText;

Everything works fine except for the highlighter. Highlighter doesn't return
me the text snippets while retrieving the results. Before this sentence
specific implementation, it worked well.

Any hints or help on this would be highly appreciated.
View this message in context:
Sent from the Lucene - Java Users mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message