db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kathey Marsden (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DERBY-5500) Native OOM error running Derby tests with default heap with IBM 32bit with 64 bit hardware
Date Thu, 10 Nov 2011 20:19:51 GMT

    [ https://issues.apache.org/jira/browse/DERBY-5500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147979#comment-13147979
] 

Kathey Marsden commented on DERBY-5500:
---------------------------------------

My understanding of this issue is that it is an issue with the default JVM heap size.   The
IBM Defaults are documented here:
http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp?topic=%2Fcom.ibm.java.doc.diagnostics.60%2Fdiag%2Fappendixes%2Fdefaults.html

On Windows the default maximum heap size is one half the real (physical memory) up to 2GB
(in practice a little less).  The JVM reserves address space for the maximum even if it is
not used.  Whatever is not used for heap is available for native memory operations such as
launching the thread in this example.

The 32 bit jvm (or any 32 bit program)  can only access 2GB even if there is more physical
memory available. The problem is that when the 32 bit jvm is run on 64 bit hardware with a
significant amount of memory, the default heap allocated is too large and does not leave enough
memory for native memory operations. For example on a 64bit 4GB machine it would take up almost
all the available memory. This can be seen by looking at the javacore file in 1.7
 
MEMUSER
1MEMUSER       JRE: 1,925,172,284 bytes / 8059 allocations
1MEMUSER       |
2MEMUSER       +--VM: 1,811,945,652 bytes / 6074 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Classes: 17,872,736 bytes / 2054 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Memory Manager (GC): 1,748,872,528 bytes / 811 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Java Heap: 1,717,567,488 bytes / 1 allocation  
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Other: 31,305,040 bytes / 810 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Threads: 41,965,092 bytes / 691 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Java Stack: 1,136,984 bytes / 153 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Native Stack: 39,878,656 bytes / 153 allocations
3MEMUSER       |  |  |
4MEMUSER       |  |  +--Other: 949,452 bytes / 385 allocations
2MEMUSER       |  |
3MEMUSER       |  +--Trace: 1,320,992 bytes / 660 allocations
2MEMUSER       |  |
3MEMUSER       |  +--JVMTI: 17,328 bytes / 13 allocations
2MEMUSER       |  |


The same JRE running on a 32 bit machine with 2GB of memory would allocate a 1GB heap leaving
plenty of room for native operations.   This issue is likely to be seen on hardware upgrade
or deploying on a newer machine where ironically adding more physical memory triggers OutOf
Memory errors.

WORKAROUND:   To work around this issue users should set their heap with the command line
option -Xmx, e.g. -Xmx1024M on the 64 bit 4GB machine would set the heap back to 1GB and leave
1GB for native memory allocations.

These are just my current interpretations. I will leave this issue open for a bit as I collect
more details and will close it as Invalid as it is a JVM bug, rather than Derby bug, but I
thought this might be a likely user hit so wanted to get the issue as I understand it and
the work-around logged.



                
> Native OOM error running Derby tests with default heap with IBM 32bit  with 64 bit hardware

> --------------------------------------------------------------------------------------------
>
>                 Key: DERBY-5500
>                 URL: https://issues.apache.org/jira/browse/DERBY-5500
>             Project: Derby
>          Issue Type: Bug
>          Components: Miscellaneous
>    Affects Versions: 10.5.3.0, 10.6.2.3, 10.7.1.4, 10.8.2.3, 10.9.0.0
>         Environment: Windows 7 64 bit 4GB memory   IBM JDK 1.6 and IBM JDK 1.7
> java version "1.6.0"
> Java(TM) SE Runtime Environment (build pwi3260sr9fp1-20110208_03(SR9 FP1))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Windows 7 x86-32 jvmwi3260sr9-20110203_74623
(JIT enabled, AOT enabled)
> J9VM - 20110203_074623
> JIT  - r9_20101028_17488ifx3
> GC   - 20101027_AA)
> JCL  - 20110203_01
> java version "1.7.0"
> Java(TM) SE Runtime Environment (build pwi3270-20110906_01)
> IBM J9 VM (build 2.6, JRE 1.7.0 Windows 7 x86-32 20110810_88604 (JIT enabled, AOT enabled)
> J9VM - R26_Java726_GA_20110810_1208_B88592
> JIT  - r11_20110810_20466
> GC   - R26_Java726_GA_20110810_1208_B88592
> J9CL - 20110810_88604)
> JCL - 20110809_01 based on Oracle 7b147
>            Reporter: Kathey Marsden
>
> With IBM JDK 1.7 
> On Windows 7 64 bit using default jvm heap values Out of Memory errors with native memory
allocation operations such as creating threads. For example:
> .........................................
> .................JVMDUMP006I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError"
- please wait.
> .... 
> 1) SetQueryTimeoutTest(org.apache.derbyTesting.functionTests.tests.jdbcapi.JDBCHarnessJavaTest)junit.framework.ComparisonFailure:
Output at line 11 expected:<Statements that should time out timed out, and statements that
should complete completed> but was:<Test SetQueryTimeoutTest FAILED>
> 	at org.apache.derbyTesting.functionTests.util.CanonTestCase.compareCanon(CanonTestCase.java:106)
> 	at org.apache.derbyTesting.functionTests.util.HarnessJavaTest.runTest(HarnessJavaTest.java:91)
> 	at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:109)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junitJVMDUMP010I Heap dump written to ....
> JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
> java.lang.OutOfMemoryError: Failed to create a thread: retVal -1073741830, errno 12
> 	at java.lang.Thread.startImpl(Native Method)
> 	at java.lang.Thread.start(Thread.java:930)
> 	at org.apache.derbyTesting.functionTests.tests.jdbcapi.SetQueryTimeoutTest.testTimeoutWithExec(SetQueryTimeoutTest.java:538)
> 	at org.apache.derbyTesting.functionTests.tests.jdbcapi.SetQueryTimeoutTest.go(SetQueryTimeoutTest.java:753)
> 	at org.apache.derbyTesting.functionTests.tests.jdbcapi.SetQueryTimeoutTest.main(SetQueryTimeoutTest.java:722)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
> 	at java.lang.reflect.Method.invoke(Method.java:611)
> 	at org.apache.derbyTesting.functionTests.util.HarnessJavaTest.runTest(HarnessJavaTest.java:87)
> 	at junit.framework.TestCase.runBare(TestCase.java:127)
> 	at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:109)
> 	at junit.framework.TestResult$1.protect(TestResult.java:106)
> 	at junit.framework.TestResult.runProtected(TestResult.java:124)
> 	at junit.framework.TestResult.run(TestResult.java:109)
> 	at junit.framework.TestCase.run(TestCase.java:118)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.framework.TestResult.runProtected(TestResult.java:124)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.framework.TestResult.runProtected(TestResult.java:124)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.framework.TestResult.runProtected(TestResult.java:124)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> 	at junit.framework.TestSuite.runTest(TestSuite.java:208)
> 	at junit.framework.TestSuite.run(TestSuite.java:203)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.framework.TestResult.runProtected(TestResult.java:124)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.framework.TestResult.runProtected(TestResult.java:124)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at junit.framework.TestSuite.runTest(TestSuite.java:208)
> 	at junit.framework.TestSuite.run(TestSuite.java:203)
> 	at junit.framework.TestSuite.runTest(TestSuite.java:208)
> 	at junit.framework.TestSuite.run(TestSuite.java:203)
> 	at junit.textui.TestRunner.doRun(TestRunner.java:116)
> 	at junit.textui.TestRunner.start(TestRunner.java:172)
> 	at junit.textui.TestRunner.main(TestRunner.java:138)
> .extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)
> 	at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
> 	at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
> 	at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
> 	at junit.extensions.TestSetup.run(TestSetup.java:23)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message