incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron McCurry (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BLUR-230) Change the index merging code to not effect the Block Cache
Date Tue, 17 Sep 2013 18:32:52 GMT
Aaron McCurry created BLUR-230:
----------------------------------

             Summary: Change the index merging code to not effect the Block Cache
                 Key: BLUR-230
                 URL: https://issues.apache.org/jira/browse/BLUR-230
             Project: Apache Blur
          Issue Type: New Feature
          Components: Blur
    Affects Versions: 0.3.0
            Reporter: Aaron McCurry
             Fix For: 0.3.0


The current implementation of the index merge scheduler will cause the Block Cache directory
to read in all the information into the cache while it's merging into a new segment.  This
may evict more useful block information and thus cause performance issues.

This task will likely have to modify the Block Cache so that if the needed data is present
in the cache it is used.  However if it's missing, it is read from HDFS and does NOT update
Block Cache.  Also needed, if the data was present in the cache have the fact the scheduler
read the data not count as a hit.  LRU caches are basically a queue with a map and every time
you read something from the map it removes the item from the queue and adds it back to the
front of the queue.  That way it's less likely to be evicted.  During merges this behavior
should be bypassed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message