lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mark harwood <>
Subject Re: Question about highlighting returning nothing
Date Thu, 16 Aug 2007 15:28:07 GMT
Highlighter deliberately returns null so the calling app can tell when the text wasn't successfully

Situations when this can happen are:

1) The text is out of synch with the index (the scenario you encountered)
2) The choice of analyzer used to tokenize the text differs from that used by the query parser
3) The document was matched on un-highlightable criteria e.g. a range query which the QueryParser
will have turned into a ConstantScoreQuery wrapping a filter for performance reasons - no
terms in the range are visible to the highlighter under these circumstances as the criteria
becomes a bitset rather than a list of terms in the rewritten query.


----- Original Message ----
From: Lukas Vlcek <>
Sent: Thursday, 16 August, 2007 4:06:36 PM
Subject: Re: Question about highlighting returning nothing


Now I understand what you are saying (seems that I had PBCAK as well ;-)

As for your last question: ...under what conditions would the highlighter
return nothing? Only if no terms matched?

I remember that I found that highlighter can return null or empty string in
different situations. I think it depends on the Analyzer used or something
like that...


On 8/16/07, Donna L Gresh <> wrote:
> Actually I don't think I'm having trouble-- as I mentioned,
> my text is *not* stored, so to do highlighting I retrieve the
> text from the database, apply the appropriate analyzer,
> and do the highlighting. It seems to be working exactly as
> it should. My problem was that in a few cases, the document
> has been removed from the database (but not from the index)
> so when I queried the database using the identifier for the "best
> hit" from the index, nothing
> was being returned. Passing "nothing" to the highlighter
> resulted in, of course, nothing, so I was getting no highlighted
> text. Once I updated my index to be in synch with the database,
> I no longer had any empty returns from the highlighter.
> Donna L. Gresh
> Services Research, Mathematical Sciences Department
> IBM T.J. Watson Research Center
> (914) 945-2472
> "Lukas Vlcek" <>
> 08/15/2007 03:49 PM
> Please respond to
> To
> cc
> Subject
> Re: Question about highlighting returning nothing
> Donna,
> I have been investigation highlighters in Lucene recently a bit. The
> humble
> experience I've learned so far is that highlighting is completely
> different
> task from indexing/searching tandem. This simple fact is not obvious to a
> lot of people. In your particular casue it would be helpful if you can
> post
> more technical details about your system settings. Not only it is
> important
> if the field to be highlighted is stored but also it is important if you
> allow for query rewrite and what king of queries you are using (Prefix,
> Wildcard ... etc).
> Just my 2 cents.
> Lukas
> On 8/15/07, Donna L Gresh <> wrote:
> >
> > Well, in my case the highlighting was returning nothing because of (my
> > favorite acronym) PBCAK--
> >
> > I don't store the text in the index, so I have to retrieve it separately
> > (from a database) for the highlighting, and my database was not in sync
> > with the index, so in a few cases the document in the index had been
> > deleted from the database--thus a score, but no document text.
> >
> > But I guess my original question remains; under what conditions would
> the
> > highlighter return nothing? Only if no terms matched?
> >
> > Donna
> >

Want ideas for reducing your carbon footprint? Visit Yahoo! For Good

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message