lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adrien Grand (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-3393) Implement an optimized LFUCache
Date Fri, 07 Sep 2012 09:53:07 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450494#comment-13450494
] 

Adrien Grand commented on SOLR-3393:
------------------------------------

bq. I agreed with his reasons. I wonder if there might be a way to have the decay happen much
less often – say after a configurable number of commits rather than for every commit.

I not comfortable with applying evictDecay based on the commit rate while cache usage depends
on the query rate: a read-only index would never benefit from it.

bq. I don't understand your first note about the put

The SolrCache.put(key, value) method should return the previous value associated with key
or null if key was not in the cache. Instead, LFUCache.put returned the value that has just
been added to the cache.

bq. On whether things should be volatile or not – I based all that on SOLR-2906, and I based
SOLR-2906 on existing stuff. I don't completely understand what the implications are. If you
do, awesome.

http://en.wikipedia.org/wiki/Volatile_variable#In_Java

One of the use-cases of the volatile keyword is to make sure that you are actually reading
the most up-to-date value of a variable. By not using this keyword, it could happen that two
CPUs don't think that a variable has the same value (because of their caches). Brian Goetz
has written a nice article on the volatile keyword in case you are interested in the features
of volatile variables http://www.ibm.com/developerworks/java/library/j-jtp06197/index.html.

We don't need it here because everything happens in synchronized blocks, which already ensure
that you are reading the most up-to-date value.


                
> Implement an optimized LFUCache
> -------------------------------
>
>                 Key: SOLR-3393
>                 URL: https://issues.apache.org/jira/browse/SOLR-3393
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Shawn Heisey
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-3393.patch, SOLR-3393.patch, SOLR-3393.patch, SOLR-3393.patch,
SOLR-3393.patch, SOLR-3393.patch
>
>
> SOLR-2906 gave us an inefficient LFU cache modeled on FastLRUCache/ConcurrentLRUCache.
 It could use some serious improvement.  The following project includes an Apache 2.0 licensed
O(1) implementation.  The second link is the paper (PDF warning) it was based on:
> https://github.com/chirino/hawtdb
> http://dhruvbird.com/lfu.pdf
> Using this project and paper, I will attempt to make a new O(1) cache called FastLFUCache
that is modeled on LRUCache.java.  This will (for now) leave the existing LFUCache/ConcurrentLFUCache
implementation in place.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message