lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lance Norskog (JIRA)" <>
Subject [jira] [Commented] (SOLR-1760) convert synonymsfilter to new tokenstream API
Date Sat, 09 Jun 2012 06:06:23 GMT


Lance Norskog commented on SOLR-1760:

This issue is obsolete.
> convert synonymsfilter to new tokenstream API
> ---------------------------------------------
>                 Key: SOLR-1760
>                 URL:
>             Project: Solr
>          Issue Type: Task
>          Components: Schema and Analysis
>            Reporter: Robert Muir
> This is the other non-trival tokenstream to convert to the new API. I looked at this
again today, and think I have a design where it will be nice and efficient.
> if you have ideas or are already looking at it, please comment!! I havent started coding
and we shouldn't duplicate any efforts.
> here is my current design:
> * add a variable 'maximumContext' to SynonymMap. This is simply the maximum singleMatch.size(),
its the maximum number of tokens lookahead that is ever needed.
> * save/restoreState/cloning can be minimized by using a stack (fixed array of maximumContext)
of references to the SynonymMap submaps. This way we can backtrack efficiently for multiword
matches without save/restoreState and less comparisons.
> * two queues (can be fixed arrays of maximumContext) are needed still for placing state
objects. the first is those that have been evaluated (always empty in the case of !preserveOriginal),
and the second is those that havent yet been evaluated, but are queued due to lookahead. 
> i plan on coding this up soon, if you have a better idea or have started work, please

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message