Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Tue, 13 Nov 2012 14:54:13 +0000 (UTC)
From: "Simon Willnauer (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: <1574492504.108096.1352818453711.JavaMail.jiratomcat@arcas>
In-Reply-To: <1280041799.108004.1352817612270.JavaMail.jiratomcat@arcas>
Subject: [jira] [Updated] (LUCENE-4556) FuzzyTermsEnum creates tons of
 objects
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


     [ https://issues.apache.org/jira/browse/LUCENE-4556?page=3Dcom.atlassi=
an.jira.plugin.system.issuetabpanels:all-tabpanel ]

Simon Willnauer updated LUCENE-4556:
------------------------------------

    Attachment: LUCENE-4556.patch

here is a patch ...scary=E2=84=A2
               =20
> FuzzyTermsEnum creates tons of objects
> --------------------------------------
>
>                 Key: LUCENE-4556
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4556
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/search, modules/spellchecker
>    Affects Versions: 4.0
>            Reporter: Simon Willnauer
>            Assignee: Simon Willnauer
>            Priority: Critical
>             Fix For: 4.1, 5.0
>
>         Attachments: LUCENE-4556.patch
>
>
> I ran into this problem in production using the DirectSpellchecker. The n=
umber of objects created by the spellchecker shoot through the roof very ve=
ry quickly. We ran about 130 queries and ended up with > 2M transitions / s=
tates. We spend 50% of the time in GC just because of transitions. Other pa=
rts of the system behave just fine here.
> I talked quickly to robert and gave a POC a shot providing a LevenshteinA=
utomaton#toRunAutomaton(prefix, n) method to optimize this case and build a=
 array based strucuture converted into UTF-8 directly instead of going thro=
ugh the object based APIs. This involved quite a bit of changes but they ar=
e all package private at this point. I have a patch that still has a fair s=
et of nocommits but its shows that its possible and IMO worth the trouble t=
o make this really useable in production. All tests pass with the patch - i=
ts a start....

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrato=
rs
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org