lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <>
Subject [jira] Commented: (LUCENE-1871) Highlighter wraps caching token filters that are not CachingTokenFilter in CachingTokenFilter
Date Fri, 28 Aug 2009 17:00:32 GMT


Uwe Schindler commented on LUCENE-1871:

I would like to have this in, because there are other use-cases of this. Maybe one TokenStream
chain is not caching on the top filter, but maybe directly after the tokenizer and before
the lowercase filter (or something like that). In this case you would also add an additional
caching to the top of the stream without really needing it.
Because of this, the marker-interface method would not do it, too, only something like isRewindable()
or isCachingTokens would do it (if Filters pass it up to their input). So as far as one filter
would cache tokens, you could detect it even on the top stream.
This is the same like the well-known InputStream usage: {code}if (!stream.markSupported) stream=new
BufferedInputStream(stream);{code}. markSupported of Filter streams also pass to their input.

I have other places, where it would be good to know if one stream supports reset() for rewinding
(or rewind(), which I would prefer). We had this discussion on java-dev some time ago, about
decoupling rewinding from resetting a stream.

> Highlighter wraps caching token filters that are not CachingTokenFilter in CachingTokenFilter
> ---------------------------------------------------------------------------------------------
>                 Key: LUCENE-1871
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: contrib/highlighter
>    Affects Versions: 2.9
>            Reporter: Mark Miller
>            Assignee: Mark Miller
>            Priority: Minor
>         Attachments: LUCENE-1871.patch
> I figured this was fine and a rare case that you would have another caching tokenstream
to feed the highlighter with - but I guess if its happening to you, especially depending on
what you are doing - its not an ideal situation.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message