lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Heisey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-3393) Implement an optimized LFUCache
Date Thu, 06 Sep 2012 22:08:07 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450096#comment-13450096
] 

Shawn Heisey commented on SOLR-3393:
------------------------------------

Adrien, thanks for looking at it and making it better.  This is early in my experience with
Java - I can still count the number of projects I've built myself on one hand.  Also, there
have been a number of changes to the entire cache system since I wrote the first patch, changes
that I have not had a chance to review.

I definitely like doing the decay only at warm time.  I'm perfectly happy to have evictDecay
yanked out.  I didn't think of the decay at all, that was Yonik on SOLR-2906.  I agreed with
his reasons.  I wonder if there might be a way to have the decay happen much less often --
say after a configurable number of commits rather than for every commit.  Also, I can't remember
whether I kept the bitshift decay (dividing by two) or changed it to subtract one from the
frequency.  IMHO subtracting one would be better.

I don't understand your first note about the put, and I can't take the time to re-read the
code right now.  On whether things should be volatile or not -- I based all that on SOLR-2906,
and I based SOLR-2906 on existing stuff.  I don't completely understand what the implications
are.  If you do, awesome.

On the default for maxFreq and how it might affect performance -- again, I expect you've got
more experience and can make a better determination.

Hoss, I would be very surprised to learn than anyone was actually using the current implementation
in 3.6.0 or the 4.0 alpha/beta.  I still haven't had a chance to give it a serious trial in
my own setup, and I wrote it!  I think about that first attempt as similar to the first sort
algorithm you ever get introduced to in a programming class, before they introduce recursion
and tell you about quicksort.  

                
> Implement an optimized LFUCache
> -------------------------------
>
>                 Key: SOLR-3393
>                 URL: https://issues.apache.org/jira/browse/SOLR-3393
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Shawn Heisey
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: SOLR-3393.patch, SOLR-3393.patch, SOLR-3393.patch, SOLR-3393.patch,
SOLR-3393.patch, SOLR-3393.patch
>
>
> SOLR-2906 gave us an inefficient LFU cache modeled on FastLRUCache/ConcurrentLRUCache.
 It could use some serious improvement.  The following project includes an Apache 2.0 licensed
O(1) implementation.  The second link is the paper (PDF warning) it was based on:
> https://github.com/chirino/hawtdb
> http://dhruvbird.com/lfu.pdf
> Using this project and paper, I will attempt to make a new O(1) cache called FastLFUCache
that is modeled on LRUCache.java.  This will (for now) leave the existing LFUCache/ConcurrentLFUCache
implementation in place.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message