accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Vines (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-3349) LruBlockCache is miscalculating sizes
Date Thu, 20 Nov 2014 19:42:36 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14219861#comment-14219861
] 

John Vines commented on ACCUMULO-3349:
--------------------------------------

So I think if we something like {code}      CachedBlock oldCb = map.put(blockName, cb);
      if (oldCb != null) {
        LOG.trace("LruBlockCache collision for block "+ blockName);
        size.addAndGet(-1 * oldCb.heapSize());
      }
{code} it can prevent the size from going off the rails

> LruBlockCache is miscalculating sizes
> -------------------------------------
>
>                 Key: ACCUMULO-3349
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3349
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.6.0
>            Reporter: John Vines
>
> On a system, we have index configured to 512MB and data block caches set to 1GB. I'm
seeing the total size seemingly horrendously miscalculated, resulting in a LOT of eviction
calls.
> {code}2014-11-17 22:07:54,814 [cache.LruBlockCache] DEBUG: Block cache LRU eviction started.
 Attempting to free 58954734976 bytes
> 2014-11-17 22:07:54,817 [cache.LruBlockCache] DEBUG: Block cache LRU eviction completed.
Freed 102696 bytes.  Priority Sizes: Single=186.14093MB (195182912), Multi=384.03125MB (402685944),Memory=0.0MB
(0)
> 2014-11-17 22:07:54,820 [cache.LruBlockCache] DEBUG: Block cache LRU eviction started.
 Attempting to free 58954734976 bytes
> 2014-11-17 22:07:54,823 [cache.LruBlockCache] DEBUG: Block cache LRU eviction completed.
Freed 102680 bytes.  Priority Sizes: Single=186.14093MB (195182920), Multi=384.03125MB (402685936),Memory=0.0MB
(0)
> 2014-11-17 22:07:54,829 [cache.LruBlockCache] DEBUG: Block cache LRU eviction started.
 Attempting to free 58954734984 bytes
> 2014-11-17 22:07:54,832 [cache.LruBlockCache] DEBUG: Block cache LRU eviction completed.
Freed 205384 bytes.  Priority Sizes: Single=186.043MB (195080224), Multi=384.12918MB (402788640),Memory=0.0MB
(0)
> {code}
> {code}2014-11-17 22:06:39,439 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=56991.715MB
(59760144624), Free=-55967.715MB (-58686402800), Max=1024.0MB
>  (1073741824), Counts: Blocks=581902, Access=261753272, Hit=202812039, Miss=58941233,
Evictions=45047347, Evicted=53777417, Ratios: Hit Ratio=77.4821341
> 0377502%, Miss Ratio=22.51785844564438%, Evicted/Run=1.1937975883483887, Duplicate Reads=4581914
> 2014-11-17 22:07:39,419 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=835.4461MB
(876028744), Free=-323.4461MB (-339157832), Max=512.0MB (536870912), Counts: Blocks=19881,
Access=171866619, Hit=171174512, Miss=692107, Evictions=300619, Evicted=595529, Ratios: Hit
Ratio=99.5972990989685%, Miss Ratio=0.402700062841177%, Evicted/Run=1.9810091257095337, Duplicate
Reads=76697
> 2014-11-17 22:07:39,439 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=56991.418MB
(59759831656), Free=-55967.418MB (-58686089832), Max=1024.0MB (1073741824), Counts: Blocks=581901,
Access=261755962, Hit=202813386, Miss=58942576, Evictions=45048521, Evicted=53778759, Ratios:
Hit Ratio=77.48185992240906%, Miss Ratio=22.518140077590942%, Evicted/Run=1.1937963962554932,
Duplicate Reads=4581916
> 2014-11-17 22:08:39,419 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=835.1727MB
(875742096), Free=-323.17273MB (-338871184), Max=512.0MB (536870912), Counts: Blocks=19762,
Access=171879228, Hit=171187080, Miss=692148, Evictions=300642, Evicted=595689, Ratios: Hit
Ratio=99.5972990989685%, Miss Ratio=0.40269438177347183%, Evicted/Run=1.9813898801803589,
Duplicate Reads=76697
> {code}
> Currently do not have log history to pinpoint where/where/if there was a slow incline
to get to 55GB or if it was spontaneous at some discernible point.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message