lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Heisey (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-2889) Implement Adaptive Replacement Cache
Date Tue, 17 Apr 2012 17:01:23 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255727#comment-13255727
] 

Shawn Heisey commented on SOLR-2889:
------------------------------------

bq. Would you mind posting some information about the results of your work and how much performance
gain you made. If you have benchmark results this would be ideal. Did you notice any increase/decrease
in memory and CPU demand?

I haven't done any extensive testing.  The testing that I did do for SOLR-2906 suggested that
the LFU cache did not offer any performance benefit over LRU, but that it didn't really cause
a performance detriment either.  I think this means that the idea was sound, but any speedups
gained from the different methodology were lost because of the basic and non-optimized implementation.

It was not a definitive test - I have two copies of my production distributed index for redundancy
purposes, with haproxy doing load balancing between the two.  I can set one set of servers
to LFU and the other to LRU, but it's production, so the two sets of servers never receive
the same queries and I don't really want to try any isolation tests on production equipment.
 My testbed is too small for a doing tests with all production data - one server with all
resources smaller than production.  I could do some tests with smaller data sets that will
fit entirely in RAM, but that will take a lot of planning that I currently don't have time
to do.

The LRU cache is highly optimized for speed, but I didn't really understand the optimizations
and they don't apply to LFU as far as I can tell.  At this time I am still using LRU cache
because I don't dare change the production configuration without authorization and I can't
leave production servers in test mode for very long.
                
> Implement Adaptive Replacement Cache
> ------------------------------------
>
>                 Key: SOLR-2889
>                 URL: https://issues.apache.org/jira/browse/SOLR-2889
>             Project: Solr
>          Issue Type: New Feature
>          Components: search
>    Affects Versions: 3.4
>            Reporter: Shawn Heisey
>            Priority: Minor
>
> Currently Solr's caches are LRU, which doesn't look at hitcount to decide which entries
are most important.  There is a method that takes both frequency and time of cache hits into
account:
> http://en.wikipedia.org/wiki/Adaptive_Replacement_Cache
> If it's feasible, this could be a good addition to Solr/Lucene.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message