lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <>
Subject [jira] [Updated] (LUCENE-5638) Default Attributes are expensive
Date Sat, 03 May 2014 16:09:18 GMT


Robert Muir updated LUCENE-5638:

    Attachment: LUCENE-5638.patch

Core tests are passing. Some analysis/common tests fail:
   [junit4] Tests with failures:
   [junit4]   - org.apache.lucene.analysis.wikipedia.WikipediaTokenizerTest.testRandomStrings
   [junit4]   - org.apache.lucene.analysis.wikipedia.WikipediaTokenizerTest.testRandomHugeStrings
   [junit4]   - org.apache.lucene.analysis.miscellaneous.TestWordDelimiterFilter.testRandomStrings
   [junit4]   - org.apache.lucene.analysis.miscellaneous.TestWordDelimiterFilter.testRandomHugeStrings
   [junit4]   - org.apache.lucene.analysis.path.TestPathHierarchyTokenizer.testRandomStrings
   [junit4]   - org.apache.lucene.analysis.path.TestPathHierarchyTokenizer.testRandomHugeStrings
   [junit4]   - org.apache.lucene.analysis.path.TestReversePathHierarchyTokenizer.testRandomStrings
   [junit4]   - org.apache.lucene.analysis.path.TestReversePathHierarchyTokenizer.testRandomHugeStrings

Looking into these now...

> Default Attributes are expensive
> --------------------------------
>                 Key: LUCENE-5638
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: modules/analysis
>            Reporter: Robert Muir
>         Attachments: LUCENE-5638.patch, LUCENE-5638.patch, LUCENE-5638.patch
> Changes like LUCENE-5634 make it clear that the default AttributeFactory stuff has a
very high cost: weakmaps/reflection/etc.
> Additionally I think clearAttributes() is more expensive than it should be: it has to
traverse a linked-list, calling clear() per token.
> Operations like cloning (save/restoreState) have a high cost tll.
> Maybe we can have a better Default? In other words, rename DEFAULT_ATTRIBUTE_FACTORY
to REFLECTION_ATTRIBUTE_FACTORY, and instead have a faster default factory that just has one
AttributeImpl with the "core ones" that 95% of users are dealing with (TOKEN_ATTRIBUTE_FACTORY?):
anything outside of that falls back to reflection.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message