commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Champagne (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CODEC-174) Improve performance of Beider Morse encoder
Date Thu, 14 Nov 2013 15:01:37 GMT

     [ https://issues.apache.org/jira/browse/CODEC-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thomas Champagne updated CODEC-174:
-----------------------------------

    Attachment: CODEC-174-convert-set-to-list-in-apply-method.patch

A new patch that converts some HashSet or LinkedHashet to LInkedList. The uniqueness in the
collection is done at the end of each applying rules.

Before patch :
{quote}
Time for encoding 80 000 times the input 'Angelo': 9 177 millis.
Time for encoding 80 000 times the input 'Angelo': 9 215 millis.
Time for encoding 80 000 times the input 'Angelo': 9 203 millis.
Time for encoding 80 000 times the input 'Angelo': 9 119 millis.
{quote}

After patch :
{quote}
Time for encoding 80 000 times the input 'Angelo': 8 096 millis.
Time for encoding 80 000 times the input 'Angelo': 8 127 millis.
Time for encoding 80 000 times the input 'Angelo': 8 158 millis.
Time for encoding 80 000 times the input 'Angelo': 8 163 millis.
{quote}

> Improve performance of Beider Morse encoder
> -------------------------------------------
>
>                 Key: CODEC-174
>                 URL: https://issues.apache.org/jira/browse/CODEC-174
>             Project: Commons Codec
>          Issue Type: Improvement
>    Affects Versions: 1.6, 1.7
>            Reporter: Thomas Champagne
>              Labels: patch, performance
>         Attachments: CODEC-174-change-rules-storage-to-Map.patch, CODEC-174-convert-set-to-list-in-apply-method.patch,
CODEC-174-delete-subsequence-cache-and-use-String.patch, CODEC-174-delete-subsequence-cache.patch,
CODEC-174-refactor-join-method-in-Phoneme.patch, CODEC-174-refactor-restrictTo-method-in-SomeLanguages.patch,
CODEC-174-reuse-set-in-PhonemeBuilder.patch, CODEC_174_cleanup.patch, TestCacheSubSequence.java,
test-commons-codec-test-bm.zip
>
>
> I use Beider Morse encoder with Solr. When it indexes a lot of documents using this encoder,
the import time is multiplied by 30. So, I have decided to optimize the current implementation
in the commons-codec.
> Currently, I have created two patch. The first patch delete a "performance hack" about
a subsequence cache. This cache doesn't optimize performance and after deleting it, you can
win some milliseconds.
> The second patch changes the storage of the rules in memory using a Map instead of List.
With it, you can access to a rule directly with the beginning of pattern. This patch divide
the encoding time by 2.
> I will try to find more improvement. If you have any idea, please tell me it.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message