lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dawid Weiss (Commented) (JIRA)" <>
Subject [jira] [Commented] (SOLR-2887) FSTSuggester shouldn't OOM on large inputs
Date Thu, 10 Nov 2011 18:40:52 GMT


Dawid Weiss commented on SOLR-2887:

No problem -- Robert is also in the need of that; nothing motivates more than real users ;)
> FSTSuggester shouldn't OOM on large inputs
> ------------------------------------------
>                 Key: SOLR-2887
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>          Components: spellchecker
>            Reporter: Dawid Weiss
>            Assignee: Dawid Weiss
>             Fix For: 3.5, 4.0
> Currently the input to FSTSuggester needs to be re-sorted and this is done in-memory.
Kind of defeats the purpose of the component since everything else is super-efficient but
we don't even get to that part because of OOMs during construction.
> Robert suggested using a spill-to-disk and merge sort on-disk. I suggested creating a
lucene index and then enumerating terms for automaton construction or taking the automaton
directly from the index structure (if it isn't pruned).

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message