lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Busch (JIRA)" <>
Subject [jira] Commented: (LUCENE-937) Make CachingTokenFilter faster
Date Fri, 22 Jun 2007 01:42:25 GMT


Michael Busch commented on LUCENE-937:

> This patch should not go through as is. What do you think given these 
> results? I assumed that an ArrayList would be faster as all of the data 
> is guaranteed contiguous, but it surprised me that the resizing was not 
> enough to slow things down to LinkedList speed (unless you start with 
> too low an initial size -- default is 10).

I think an ArrayList also has higher initialization costs. Your test
actually tests the performance for a single document. It would be 
interesting to know how the different implementations perform when you 
run the tests with more than one document. I would think that LinkedList() 
is probably better if you have lots of very small documents, whereas
ArrayList(30) is faster if you have bigger docs with lots of Tokens.

> Make CachingTokenFilter faster
> ------------------------------
>                 Key: LUCENE-937
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Mark Miller
>            Priority: Minor
>         Attachments: CachingTokenFilter.patch
> The wrong data structure was used for the CachingTokenFilter. It should be an ArrayList
rather than a LinkedList. There is a noticeable difference in speed.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message