lucene-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bruno Roustant (Jira)" <j...@apache.org>
Subject [jira] [Assigned] (LUCENE-9064) Can we remove the FST cache in Kuromoji and Nori analyzers?
Date Mon, 02 Dec 2019 09:37:00 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-9064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bruno Roustant reassigned LUCENE-9064:
--------------------------------------

    Assignee: Bruno Roustant

> Can we remove the FST cache in Kuromoji and Nori analyzers?
> -----------------------------------------------------------
>
>                 Key: LUCENE-9064
>                 URL: https://issues.apache.org/jira/browse/LUCENE-9064
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Bruno Roustant
>            Assignee: Bruno Roustant
>            Priority: Minor
>
> Is the ~30k han cache in kuromoji redundant after LUCENE-8920?
> [https://github.com/apache/lucene-solr/blob/813ca77250db29116812bc949e2a466a70f969a3/lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/dict/TokenInfoFST.java#L35-L38])
> The entire linked file's purpose is all around this caching, so if its not needed anymore
it would be a nice cleanup. But it was definitely needed for good performance before, so we
shoudl be careful. Nori analyzer has the exact same thing (file has the same name) for ~10k
hangul syllables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Mime
View raw message