lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Heisey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-3393) Implement an optimized LFUCache
Date Mon, 23 Apr 2012 13:59:36 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13259609#comment-13259609
] 

Shawn Heisey commented on SOLR-3393:
------------------------------------

I've been working on this.  I've come to realize that I don't completely understand how CacheRegenerator
works.  I suspect that it is geared around LRU caches and that the new cache won't have any
of the frequency information from the old one, it will just put the entries into the cache
as if they were new.  Can anyone confirm this?  If I am right, I think my SOLR-2906 implementation
is incorrect at warm time.

After the new cache is regenerated, should I go through the new cache, grab the frequency
information from the old cache with each key, and fix the new cache up?

Yonik, you were the one that came up with the timeDecay option for SOLR-2906.  It was added
to the markAndSweep routine (which evicts old entries).  Should it also be in the warm routine,
or possibly only exist in the warm routine?

                
> Implement an optimized LFUCache
> -------------------------------
>
>                 Key: SOLR-3393
>                 URL: https://issues.apache.org/jira/browse/SOLR-3393
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 3.6, 4.0
>            Reporter: Shawn Heisey
>            Priority: Minor
>             Fix For: 4.0
>
>
> SOLR-2906 gave us an inefficient LFU cache modeled on FastLRUCache/ConcurrentLRUCache.
 It could use some serious improvement.  The following project includes an Apache 2.0 licensed
O(1) implementation.  The second link is the paper (PDF warning) it was based on:
> https://github.com/chirino/hawtdb
> http://dhruvbird.com/lfu.pdf
> Using this project and paper, I will attempt to make a new O(1) cache called FastLFUCache
that is modeled on LRUCache.java.  This will (for now) leave the existing LFUCache/ConcurrentLFUCache
implementation in place.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message