incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron McCurry (JIRA)" <>
Subject [jira] [Commented] (BLUR-229) Make V2 of the Block Cache production ready.
Date Thu, 10 Oct 2013 16:47:42 GMT


Aaron McCurry commented on BLUR-229:

I think that this is getting pretty close.;a=commit;h=28470cd67e632a476025ce91d3a1f2b8e243f687;a=commit;h=7fa3a12098500e79317ab835f1bce940631a0a1d

Once a couple of burn-in tests are finished I think we can deem this complete.

> Make V2 of the Block Cache production ready.
> --------------------------------------------
>                 Key: BLUR-229
>                 URL:
>             Project: Apache Blur
>          Issue Type: New Feature
>          Components: Blur
>    Affects Versions: 0.3.0, 0.2.1
>            Reporter: Aaron McCurry
>             Fix For: 0.3.0, 0.2.1
> This task will be to make the new version of the Block Cache production ready.
> The code can be found in:
> The big features for this new code are the following:
> 1. Variable length cache block sizes.  The big need for this is in filtering.  Creating
huge bits sets on the heap to cache filters created by inbound queries can be hard to manage.
 It can also production performance problems (GC issues) and can cause OOM errors if too many
filters are created.  If a filter can be written as a file per segment and cached as a single
unit in the block cache, then it will be as performant as on heap bitset but will live off
the heap and under the same resource constraints placed on the block cache.  So basically
when a cluster is over allocated (when it comes to caching filters) the cluster will be less
likely to fail and instead will perform more slowly.
> 2. Block cache size can be dynamically changed.
> 3. Simpler implementation for the off heap slabs of memory.  Which should also make it

This message was sent by Atlassian JIRA

View raw message