lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <>
Subject [jira] Commented: (LUCENE-1040) Can't quickly create StopFilter
Date Thu, 01 Nov 2007 21:10:51 GMT


Hoss Man commented on LUCENE-1040:

> But it does if the Set is not a CharArraySet.
> Docs should be clearer though that ignoreCase is ignored if passing a CharArraySet (or
is there a more sane way?)

Hmmm... actually .. does making the CharArraySet capable of doing case insensitive comparisons
over complicate things?  perhaps the CharArraySet should just be a set of strings that supports
fast lookup lookup of strings, and should be responsible for lowercasing the
terms before doing the set lookup (and makeStopSet should be responsible for lowercasing hte
tersm before putting them in the set).  it would probably be good to change the sigs to something
like StopFilter(TokenStream s, Set stopWords, boolean lowerCaseBeforeLookup) and makeStopSet(String[]
stopWords, boolean lowerCaseBeforeAdding) so it was clear what expectations StopFilter has
on the set if people make one from scratch.

this is kind of an orthogonal API discussion to the CharArraySet issue though ... the same
arguments could be made about the 2.2 instance of StopFilter ... i just bring it up as a potential
point of confusion since people could expect these two work the same way, and they wont (unless
i'm missing something)...

   Set a = ...; // something with lots of mixed case words
   Set b = new CharArraySet(a, false);
   StopFilter aaa = new StopFilter(stream, a, true)
   StopFilter bbb = new StopFilter(stream, b, true)

> Can't quickly create StopFilter
> -------------------------------
>                 Key: LUCENE-1040
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Yonik Seeley
>            Assignee: Yonik Seeley
>         Attachments: CharArraySet.patch, CharArraySet.take2.patch
> Due to the use of CharArraySet by StopFilter, one can no longer efficiently pre-create
a Set for use by future StopFilter instances.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message