db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mamta A. Satoor (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (DERBY-4200) client side OutOfMemoryError running derbnetclientmats:jdbcapi/derbyStress
Date Fri, 16 Oct 2009 05:30:31 GMT

    [ https://issues.apache.org/jira/browse/DERBY-4200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766122#action_12766122
] 

Mamta A. Satoor edited comment on DERBY-4200 at 10/15/09 10:29 PM:
-------------------------------------------------------------------

On Aug 13th comment, I mentioned trying dumps **after** we get out of 2000 iterations of executing
the same query repeatedly. Looking back at the OOM failure, we get exception within the loop.
Based on that, I changed the testDerby3316() in derbyStress as follows. In the code below,
I added the dumps within the loop every 100 iterations rather than one dump outside of the
loop. 
    public static void testDerby3316() throws Exception {
          System.out.println("DERBY-3316: Multiple statement executions ");
          Connection conn = ij.startJBMS();
                
          Statement s = conn.createStatement();
          s.executeUpdate("CREATE TABLE TAB (col1 varchar(32672))");
          PreparedStatement ps = conn.prepareStatement("INSERT INTO TAB VALUES(?)");
          ps.setString(1,"hello");
          ps.executeUpdate();
          ps.setString(1,"hello");
          ps.executeUpdate();
          ps.close();
          for (int i = 0; i < 2000; i++)
          {
                  s = conn.createStatement();
                  ResultSet rs = s.executeQuery("SELECT * from tab");
                  if (i%100 == 0) {
                	  System.out.println("Iteration number " + i);
                      System.gc();
//                      System.runFinalization();
                      com.ibm.jvm.Dump.HeapDump();
                      com.ibm.jvm.Dump.JavaDump();

                  }
                  // drain the resultset
                  while (rs.next());
                  // With DERBY-3316, If I don't explicitly close the resultset or 
                  // statement, we get a leak.
                  //rs.close();
                  //s.close();
          }    
          // close the connection to free up all the result sets that our sloppy 
          // user didn't close.
          conn.close();
          conn = ij.startJBMS();
          s = conn.createStatement();
          s.executeUpdate("DROP TABLE TAB");
          s.close();
          conn.close();
       }

With the code above, when I run it on my windows XP machine with ibm15, the number of org/apache/derby/client/net/NetStatement
range from 2 to 102. But the results on vmware is that the number of org/apache/derby/client/net/NetStatement
keep increasing in the dump every 100 iterations. It starts with about 4 objects and keeps
going up by 100 and right about when the number of objects get to 940+ or so, it runs in OOM.
The OOM errors are intermittent but whenever I saw them, it was always when it got to about
that 900+ number.

Talked to Mike about this behavior and he suggested that may be I try to put a sleep after
the gc because depending on the system/vc, a call to gc may just schedule a background gc
and not necessarily mean objects are being claimed when the call to gc returns. I will get
that gc/sleep combo a try and see how that goes.


      was (Author: mamtas):
    On Aug 13th comment, I mentioned trying dumps **after** we get out of 2000 iterations
of executing the same query repeatedly. Looking back at the OOM failure, we get exception
within the loop. Based on that, I changed the testDerby3316() in derbyStress as follows. In
the code below, I added the dumps within the loop every 100 iterations rather than one dump
outside of the loop. 
    public static void testDerby3316() throws Exception {
          System.out.println("DERBY-3316: Multiple statement executions ");
          Connection conn = ij.startJBMS();
                
          Statement s = conn.createStatement();
          s.executeUpdate("CREATE TABLE TAB (col1 varchar(32672))");
          PreparedStatement ps = conn.prepareStatement("INSERT INTO TAB VALUES(?)");
          ps.setString(1,"hello");
          ps.executeUpdate();
          ps.setString(1,"hello");
          ps.executeUpdate();
          ps.close();
          for (int i = 0; i < 2000; i++)
          {
                  s = conn.createStatement();
                  ResultSet rs = s.executeQuery("SELECT * from tab");
                  if (i%100 == 0) {
                	  System.out.println("Iteration number " + i);
                      System.gc();
//                      System.runFinalization();
                      com.ibm.jvm.Dump.HeapDump();
                      com.ibm.jvm.Dump.JavaDump();

                  }
                  // drain the resultset
                  while (rs.next());
                  // With DERBY-3316, If I don't explicitly close the resultset or 
                  // statement, we get a leak.
                  //rs.close();
                  //s.close();
          }    
          // close the connection to free up all the result sets that our sloppy 
          // user didn't close.
          conn.close();
          conn = ij.startJBMS();
          s = conn.createStatement();
          s.executeUpdate("DROP TABLE TAB");
          s.close();
          conn.close();
       }

With the code above, when I run it on my windows XP machine with ibm16, the number of org/apache/derby/client/net/NetStatement
range from 2 to 40. But the results on vmware is that the number of org/apache/derby/client/net/NetStatement
keep increasing in the dump every 100 iterations. It starts with about 4 objects and keeps
going up by 100 and right about when the number of objects get to 940+ or so, it runs in OOM.
The OOM errors are intermittent but whenever I saw them, it was always when it got to about
that 900+ number.

Talked to Mike about this behavior and he suggested that may be I try to put a sleep after
the gc because depending on the system/vc, a call to gc may just schedule a background gc
and not necessarily mean objects are being claimed when the call to gc returns. I will get
that gc/sleep combo a try and see how that goes.

  
> client side OutOfMemoryError running derbnetclientmats:jdbcapi/derbyStress
> --------------------------------------------------------------------------
>
>                 Key: DERBY-4200
>                 URL: https://issues.apache.org/jira/browse/DERBY-4200
>             Project: Derby
>          Issue Type: Bug
>          Components: Network Client
>    Affects Versions: 10.5.2.0
>         Environment: java version "1.5.0"
> Java(TM) 2 Runtime Environment, Standard Edition (build pxi32devifx-20070806 (SR5a))
> IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux x86-32 j9vmxi3223-20070426 (JIT enabled)
> J9VM - 20070420_12448_lHdSMR
> JIT  - 20070419_1806_r8
> GC   - 200704_19)
> JCL  - 20070725
> SUSE linux running on vmware.
>            Reporter: Kathey Marsden
>         Attachments: heapdump.20090428.084024.25679.phd, javacore.20090428.084024.25679.txt,
vmware10_5_SR10HeapDump.phd, vmware10_5_SR7HeapDump.phd, windows10_5_HeapDump.phd
>
>
> On the nightly run for 4/27 - 10.5.1.2 - (769232), I saw client jdbcapi/derbystress.java
 run out of heap space.   The test has not failed like this before on the same machine with
the same JVM, and the one checkin on that day DERBY-3991 could not account for this failure.
> I will attach the javacore and heapdump.  Taking a quick look at the heap dump, it seems
to have a lot of client side Statement objects, which seems to be just the leak the test is
checking for.  Note: the test runs with 64MB heap.  It would be interesting to run with other
jvms and force a gc() and a heap dump at this point in the test and see if we still have a
lot of Statement objects or if this is a specific platform/JVM issue.
> The trace at the time of failure was :
> 1XMCURTHDINFO  Current Thread Details
> NULL           ----------------------
> 3XMTHREADINFO      "main" (TID:0x0808D300, sys_thread_t:0x0805CBC8, state:R, native ID:0x0000644F)
prio=5
> 4XESTACKTRACE          at org/apache/derby/client/am/Cursor.allocateCharBuffer(Bytecode
PC:77(Compiled Code))
> 4XESTACKTRACE          at org/apache/derby/client/net/NetStatementReply.parseSQLDTARDarray(Bytecode
PC:77(Compiled Code))
> 4XESTACKTRACE          at org/apache/derby/client/net/NetStatementReply.parseQRYDSC(Bytecode
PC:10(Compiled Code))
> 4XESTACKTRACE          at org/apache/derby/client/net/NetStatementReply.parseOpenQuery(Bytecode
PC:104(Compiled Code))
> 4XESTACKTRACE          at org/apache/derby/client/net/NetStatementReply.parseOPNQRYreply(Bytecode
PC:14(Compiled Code))
> 4XESTACKTRACE          at org/apache/derby/client/net/NetStatementReply.readOpenQuery(Bytecode
PC:6(Compiled Code))
> 4XESTACKTRACE          at org/apache/derby/client/net/StatementReply.readOpenQuery(Bytecode
PC:7(Compiled Code))
> 4XESTACKTRACE          at org/apache/derby/client/net/NetStatement.readOpenQuery_(Bytecode
PC:11(Compiled Code))
> 4XESTACKTRACE          at org/apache/derby/client/am/Statement.readOpenQuery(Bytecode
PC:6(Compiled Code))
> 4XESTACKTRACE          at org/apache/derby/client/am/Statement.flowExecute(Bytecode PC:581(Compiled
Code))
> 4XESTACKTRACE          at org/apache/derby/client/am/Statement.executeQueryX(Bytecode
PC:3(Compiled Code))
> 4XESTACKTRACE          at org/apache/derby/client/am/Statement.executeQuery(Bytecode
PC:3(Compiled Code))
> 4XESTACKTRACE          at org/apache/derbyTesting/functionTests/tests/jdbcapi/derbyStress.testDerby3316(derbyStress.java:156)
> 4XESTACKTRACE          at org/apache/derbyTesting/functionTests/tests/jdbcapi/derbyStress.main(derbyStress.java:57(Compiled
Code))

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message