accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-3349) LruBlockCache is miscalculating sizes
Date Tue, 14 Apr 2015 02:24:12 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493458#comment-14493458
] 

Josh Elser commented on ACCUMULO-3349:
--------------------------------------

Thanks for committing this, John! I noticed that you didn't include any new tests. Have you
run with your changes locally to verify that the cache reports more reasonable values over
time?

If this really was common enough in every deployment and we evicted blocks early frequently
(or never cached certain blocks), this could have a big (positive) impact on performance.
It would be good to have a better feel for the net-effect of your change.

> LruBlockCache is miscalculating sizes
> -------------------------------------
>
>                 Key: ACCUMULO-3349
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3349
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.6.0
>            Reporter: John Vines
>            Assignee: John Vines
>             Fix For: 1.7.0
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> On a system, we have index configured to 512MB and data block caches set to 1GB. I'm
seeing the total size seemingly horrendously miscalculated, resulting in a LOT of eviction
calls.
> {code}2014-11-17 22:07:54,814 [cache.LruBlockCache] DEBUG: Block cache LRU eviction started.
 Attempting to free 58954734976 bytes
> 2014-11-17 22:07:54,817 [cache.LruBlockCache] DEBUG: Block cache LRU eviction completed.
Freed 102696 bytes.  Priority Sizes: Single=186.14093MB (195182912), Multi=384.03125MB (402685944),Memory=0.0MB
(0)
> 2014-11-17 22:07:54,820 [cache.LruBlockCache] DEBUG: Block cache LRU eviction started.
 Attempting to free 58954734976 bytes
> 2014-11-17 22:07:54,823 [cache.LruBlockCache] DEBUG: Block cache LRU eviction completed.
Freed 102680 bytes.  Priority Sizes: Single=186.14093MB (195182920), Multi=384.03125MB (402685936),Memory=0.0MB
(0)
> 2014-11-17 22:07:54,829 [cache.LruBlockCache] DEBUG: Block cache LRU eviction started.
 Attempting to free 58954734984 bytes
> 2014-11-17 22:07:54,832 [cache.LruBlockCache] DEBUG: Block cache LRU eviction completed.
Freed 205384 bytes.  Priority Sizes: Single=186.043MB (195080224), Multi=384.12918MB (402788640),Memory=0.0MB
(0)
> {code}
> {code}2014-11-17 22:06:39,439 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=56991.715MB
(59760144624), Free=-55967.715MB (-58686402800), Max=1024.0MB
>  (1073741824), Counts: Blocks=581902, Access=261753272, Hit=202812039, Miss=58941233,
Evictions=45047347, Evicted=53777417, Ratios: Hit Ratio=77.4821341
> 0377502%, Miss Ratio=22.51785844564438%, Evicted/Run=1.1937975883483887, Duplicate Reads=4581914
> 2014-11-17 22:07:39,419 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=835.4461MB
(876028744), Free=-323.4461MB (-339157832), Max=512.0MB (536870912), Counts: Blocks=19881,
Access=171866619, Hit=171174512, Miss=692107, Evictions=300619, Evicted=595529, Ratios: Hit
Ratio=99.5972990989685%, Miss Ratio=0.402700062841177%, Evicted/Run=1.9810091257095337, Duplicate
Reads=76697
> 2014-11-17 22:07:39,439 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=56991.418MB
(59759831656), Free=-55967.418MB (-58686089832), Max=1024.0MB (1073741824), Counts: Blocks=581901,
Access=261755962, Hit=202813386, Miss=58942576, Evictions=45048521, Evicted=53778759, Ratios:
Hit Ratio=77.48185992240906%, Miss Ratio=22.518140077590942%, Evicted/Run=1.1937963962554932,
Duplicate Reads=4581916
> 2014-11-17 22:08:39,419 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=835.1727MB
(875742096), Free=-323.17273MB (-338871184), Max=512.0MB (536870912), Counts: Blocks=19762,
Access=171879228, Hit=171187080, Miss=692148, Evictions=300642, Evicted=595689, Ratios: Hit
Ratio=99.5972990989685%, Miss Ratio=0.40269438177347183%, Evicted/Run=1.9813898801803589,
Duplicate Reads=76697
> {code}
> Currently do not have log history to pinpoint where/where/if there was a slow incline
to get to 55GB or if it was spontaneous at some discernible point.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message