accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-3549) tablet server location cache may grow too large
Date Fri, 30 Jan 2015 22:58:34 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14299331#comment-14299331
] 

Keith Turner commented on ACCUMULO-3549:
----------------------------------------

I wrote a little test to simulate memory usage, seeing 265 bytes per cache entry for 10 byte
end rows.

{code:java}
  public static void main(String[] args) throws Exception {
    TreeMap<Text,TabletLocation> metaCache = new TreeMap<Text,TabletLocation>(endRowComparator);

    Text per = null;

    Runtime rt = Runtime.getRuntime();

    for (int i = 0; i < 1000000; i++) {
      Text er = new Text("ABC_" + String.format("%06d", i));

      KeyExtent ke = new KeyExtent(new Text("5"), er, per);
      per = er;

      String loc = "192.168.1." + (i % 100);

      metaCache.put(er, new TabletLocation(ke, loc, String.format("%x", loc.hashCode())));
    }

    long used = rt.totalMemory() - rt.freeMemory();

    while (true) {
      System.gc();
      Thread.sleep(1000);

      long used2 = rt.totalMemory() - rt.freeMemory();
      if (used2 / 1000000 == used / 1000000)
        break;

      used = used2;
    }

    used = rt.totalMemory() - rt.freeMemory();
    System.out.println(String.format("%,d", used));
    System.out.println(String.format("%.2f", used / (double) metaCache.size()));
  }
{code}



> tablet server location cache may grow too large
> -----------------------------------------------
>
>                 Key: ACCUMULO-3549
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-3549
>             Project: Accumulo
>          Issue Type: Bug
>          Components: tserver
>            Reporter: Eric Newton
>            Priority: Minor
>             Fix For: 1.7.0
>
>
> Now that we're no longer clearing the location cache in the tablet server, it could grow
without bound.  It should be cleared, either by time, or using an LRU.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message