Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Received-SPF: pass (athena.apache.org: local policy)
Message-ID: <4E5B73C0.1050503@oracle.com>
Date: Mon, 29 Aug 2011 13:10:56 +0200
From: Kristian Waagan <kristian.waagan@oracle.com>
User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US;
 rv:1.9.2.15) Gecko/20110412 Thunderbird/3.1.9
MIME-Version: 1.0
To: Derby Development <derby-dev@db.apache.org>
Subject: Problems with junit-lowmem
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

Some new tests have recently been added to the low-memory test suite 
that we have (you can run it with 'ant junit-lowmem'). I'm now seeing 
two problems:
  a) I get an OOME now and then
  b) It takes a long time to complete

Obviously, this looks like a memory leak in Derby. That's also pretty 
much correct, but it's what I'll call a "controlled memory leak". I 
believe the test I added for DERBY-4137 [1] is causing both (a) and (b). 
It worked fine before the latest test additions, but now the memory 
pressure is too high. The memory occupied for the 120'000 transactions 
amounted to around 4 MB.

In DERBY-4137, the use of a timer causes memory leaks. The leak will 
"self-correct" when the specified timeout duration has passed. In the 
test, this value has been set very high and there is no guarantee this 
won't happen in real-world applications either. In real-world 
applications the size of the memory leak is given by transaction rate 
and timeout value (the higher the tx rate and the longer the timeout 
value, the more memory is hogged).
The fix only reduced the amount of memory leaked. In Java 5.0 there's a 
method that can be used to purge the queue, but using it seems to be a 
little bit complicated (i.e. when to invoke it). In any case, it will be 
easier to deal with this once we have sunset support for J2SE 1.4 [2].

I can think of the following actions to reduce the likelihood of this 
test causing problems:
  1) Run the test only as embedded (reduces memory usage to ~half)
  2) Force execution order and run this test as the last test (this 
fixes the problem today, but it may reoccur if/when another test eats up 
more memory)
  3) Reduce the timeout value such that the data can be garbage 
collected shortly after the test has been run (timing/performance sensitive)

Further, the following actions remove the problem altogether:
  i)   Delete the test.
  ii)  Run test in a separate VM.
  iii) Fix issue using Java 5.0 API.
       (some details to consider here)


For starters I will implement the changes suggested in (1) and (2). I 
will test with only (2) first to see if we are already at a point where 
the tests require more memory than what is available. See DERBY-5394 [3].


-- 
Kristian

[1] https://issues.apache.org/jira/browse/DERBY-4137.
[2] If this code is to be run on small platforms, we're back to square 
one. It's not a problem technically, the code will just be a lot more 
verbose and complex due to the use of reflection.
[3] https://issues.apache.org/jira/browse/DERBY-5394.