hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1554) TestHeapSize failing on Hudson
Date Mon, 29 Jun 2009 18:22:47 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725301#action_12725301

Jonathan Gray commented on HBASE-1554:

I actually didn't know my new block cache was going to get committed yet.  It was done and
tested but I knew there was a heapsize issue with it.

There's a mismatch between how the LRU is tracking it's heap and how ClassSize is calculating

Since the LRU pre-allocates it's backing map, I include all of that overhead in the initial
overhead of the LRU.  ClassSize does not consider that memory, only the reference to the map.

There are a large number of heapSize tests in the LRU unit tests, almost all of them at some
point verify that it's properly keeping track of its size.  What is not done in those tests
is verifying that the initial overhead/heapsize is perfectly aligned with the exact overhead
of the object (and backing map).... However, since there is A) only one of these per regionserver,
and B) the overhead from the Map is more than two orders of magnitude larger than the overhead
of the LRU object itself... I don't think this is a particularly big deal.

I will work with Erik on figuring this out.  We should do _some_ kind of test that will ensure
if we make changes to the LRU without updating the overhead, a unit test will fail.  That
either means a special constructor, or a different style of unit test that does a deeper analysis....
maybe both.

> TestHeapSize failing on Hudson
> ------------------------------
>                 Key: HBASE-1554
>                 URL: https://issues.apache.org/jira/browse/HBASE-1554
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: Andrew Purtell
>            Assignee: Erik Holstad
>            Priority: Critical
>             Fix For: 0.20.0
> TestHeapSize worked for me when tested locally but failing up on Hudson:
> {code}
>     [junit] Running org.apache.hadoop.hbase.io.TestHeapSize
>     [junit] 2009-06-20 01:29:11,577 DEBUG [main] util.ClassSize(121): bytes
>     [junit] 	class [B
>     [junit] 2009-06-20 01:29:11,580 DEBUG [main] util.ClassSize(121): offset
>     [junit] 	int
>     [junit] 2009-06-20 01:29:11,580 DEBUG [main] util.ClassSize(121): length
>     [junit] 	int
>     [junit] 2009-06-20 01:29:11,581 DEBUG [main] util.ClassSize(147): Primitives 8, arrays
1, references(inlcuding 16, for object overhead) 2, refSize 4, size 32
>     [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.181 sec
>     [junit] Test org.apache.hadoop.hbase.io.TestHeapSize FAILED
> {code}

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message