incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron McCurry (JIRA)" <>
Subject [jira] [Commented] (BLUR-230) Change the index merging code to not effect the Block Cache
Date Sat, 12 Oct 2013 11:47:42 GMT


Aaron McCurry commented on BLUR-230:

Yes it is.  Sorry about that, good catch.


> Change the index merging code to not effect the Block Cache
> -----------------------------------------------------------
>                 Key: BLUR-230
>                 URL:
>             Project: Apache Blur
>          Issue Type: New Feature
>          Components: Blur
>    Affects Versions: 0.3.0
>            Reporter: Aaron McCurry
>             Fix For: 0.3.0
> The current implementation of the index merge scheduler will cause the Block Cache directory
to read in all the information into the cache while it's merging into a new segment.  This
may evict more useful block information and thus cause performance issues.
> This task will likely have to modify the Block Cache so that if the needed data is present
in the cache it is used.  However if it's missing, it is read from HDFS and does NOT update
Block Cache.  Also needed, if the data was present in the cache have the fact the scheduler
read the data not count as a hit.  LRU caches are basically a queue with a map and every time
you read something from the map it removes the item from the queue and adds it back to the
front of the queue.  That way it's less likely to be evicted.  During merges this behavior
should be bypassed.

This message was sent by Atlassian JIRA

View raw message