lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Heisey (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-2906) Implement LFU Cache
Date Fri, 09 Dec 2011 18:26:40 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166399#comment-13166399
] 

Shawn Heisey commented on SOLR-2906:
------------------------------------

I finally got a chance to do some testing in production.  I have two distributed index chains,
both running 3.5.0 with this patch and the one from SOLR-1972 applied.  The chains are updated
independently, there is no replication.  It's not a truly definitive test, because my queries
are load balanced between the two chains and I do have some hardware discrepancies.  I cannot
create a valid test environment to compare the two caches as they should be compared, with
identical queries going to two completely identical servers.

On average, commits made on chain A where filterCache is set to LFU and the servers have 48GB
of RAM happen faster than those on chain B, with FastLRU and 64GB of RAM.  One of the two
servers on chain A has slightly faster processors than its counterpart on chain B -- 2.83
GHz vs. 2.66 GHz.  The other two servers on both chains have 2.5 GHz processors.

I suspect that most of the potential gains that the LFU algorithm might be able to provide
are swallowed by the very inefficient implementation.

If anyone has some thoughts for me to pursue, I will be happy to do so, but I am out of my
own ideas.  I hope the patch will be committed.  It could use a lot of optimization and there's
probably cosmetic cleanup to do.

                
> Implement LFU Cache
> -------------------
>
>                 Key: SOLR-2906
>                 URL: https://issues.apache.org/jira/browse/SOLR-2906
>             Project: Solr
>          Issue Type: Sub-task
>          Components: search
>    Affects Versions: 3.4
>            Reporter: Shawn Heisey
>            Priority: Minor
>         Attachments: ConcurrentLFUCache.java, LFUCache.java, SOLR-2906.patch, SOLR-2906.patch,
TestLFUCache.java
>
>
> Implement an LFU (Least Frequently Used) cache as the first step towards a full ARC cache

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message