db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deepa Remesh (JIRA)" <derby-...@db.apache.org>
Subject [jira] Updated: (DERBY-210) Network Server will leak prepared statements if not explicitly closed by the user until the connection is closed
Date Thu, 02 Mar 2006 01:36:42 GMT
     [ http://issues.apache.org/jira/browse/DERBY-210?page=all ]

Deepa Remesh updated DERBY-210:
-------------------------------

    Attachment: derby-210-patch5-v1.diff
                derby-210-patch5-v1.status

Attaching a patch 'derby-210-patch5-v1.diff' which removes the actual memory leaks. The previous
patches were doing preliminary work to fix the client-side finalization and network server
object re-use. This patch removes the actual memory leaks.

-----------------------------------------------------------------
Summary of patch:
-----------------------------------------------------------------
1. Eliminates the below references to PreparedStatement objects by using WeakHashMap instead
of LinkedList. When there are no other references to the keys in a WeakHashMap, they will
get removed from the map and can thus get garbage-collected. They do not have to wait till
the Connection object is collected.
       - 'openStatements_' in org.apache.derby.client.am.Connection
       - 'CommitAndRollbackListeners_' in org.apache.derby.client.am.Connection

2. Updates the following comment for openStatements_:
// Since DERBY prepared statements must be re-prepared after a commit,
// then we must traverse this list after a commit and notify statements
// that they are now in an un-prepared state.
final java.util.LinkedList openStatements_ = new java.util.LinkedList();

In the code, I did not see this list being traversed after a commit to re-prepare statements.
Also, I think this is not needed since Derby does not require re-prepare of statements after
a commit. Currently, this list is used to close all open statements when the originating connection
is closed.

3. Removes all ResultSets from HashTable 'positionedUpdateCursorNameToResultSet_' in SectionManager.
Only result sets of positioned update statements were being removed from this hashtable whereas
all result sets were added. Because of this, client driver was holding on to result sets and
statements even after rs.close() was called.

4. Modifies the test jdbcapi/derbyStress.java to run the test for derby-210. The test was
checked in as patch2  but disabled for client framework.

-----------------------------------------------------------------
Open items related to this issue
-----------------------------------------------------------------
* http://issues.apache.org/jira/browse/DERBY-817 (Improvements to Network Client driver -
analyze/improve use of java collection classes in the code) - This issue lists some potential
problem areas and cleanup tasks identified in the client driver code.

* http://issues.apache.org/jira/browse/DERBY-1002 (Check that DRDAStatement and DRDAResultSet
states are reset when they are re-used) - This issue was blocking DERBY-210. Patch for the
blocking issue has been committed. There is still some minor cleanup to be done in this issue.

* http://issues.apache.org/jira/browse/DERBY-1021 (Perform cleanup actions which require synchronization
outside the finalizer) - This issue is linked to ("is part of") DERBY-210. Maybe, it is easier
if I had made this a sub-task of DERBY-210. I will close DERBY-1021 and open a sub-task for
210 with details about work needed on the finalizer.

With this patch, I ran the following tests on Windows XP:
* derbynetclientmats suite 6 times on my machine (3 times with Sun JDK1.4.2 and 3 times with
Sun JDK1.5). No failures. 

* repro with 100,000 statements and checked the memory usage on client and server. No spikes
in memory usage. Without this patch, the repro used to give OutOfMemory error after executing
950 statements. Tests run with Sun JDK1.4.2 with 64M heap space .

* test to compare memory usage when statements_are_not_explicitly_closed vs statements_are_explicitly_closed.
Tests run with Sun JDK1.4.2 with 64M heap space.
   case1: repro with 5000 statements 
   case2: modified_repro_which_explicitly_closes_prepared_statements with 5000 statements.

I compared the memory usage on client and server in these two cases and saw that memory usage
over time is similar. Memory usage is calculated as Runtime.totalMemory() - Runtime.freeMemory().

I would appreciate if someone can take a look at this patch.

> Network Server will leak prepared statements if not explicitly closed by the user until
the connection is closed
> ----------------------------------------------------------------------------------------------------------------
>
>          Key: DERBY-210
>          URL: http://issues.apache.org/jira/browse/DERBY-210
>      Project: Derby
>         Type: Bug
>   Components: Network Client
>     Reporter: Kathey Marsden
>     Assignee: Deepa Remesh
>  Attachments: DOTS_ATCJ2_Derby-noPatch.png, DOTS_ATCJ2_Derby-withPatch.png, derby-210-patch1.diff,
derby-210-patch2.diff, derby-210-patch2.status, derby-210-patch3.diff, derby-210-patch4-v2.diff,
derby-210-patch4-v2.status, derby-210-patch4-v3.diff, derby-210-patch4-v3.status, derby-210-patch5-v1.diff,
derby-210-patch5-v1.status, derby-210-v2-draft.diff, derby-210-v2-draft.status, derbyStress.java
>
> Network server will not garbage collect prepared statements that are not explicitly closed
by the user.  So  a loop like this will leak.
> ...
> PreparedStatement ps;
>  for (int i = 0 ; i  < numPs; i++)
> 	{
> 	 ps = conn.prepareStatement(selTabSql);
> 	 rs =ps.executeQuery();
> 	 while (rs.next())
> 	{
> 	    rs.getString(1);
> 	}
> 	rs.close();
> 	// I'm a sloppy java programmer
> 	//ps.close();
> 	}
> 			
> To reproduce run the attached program 
> java derbyStress
> Both client and server will grow until the connection is closed.
>  
> It is likely that the fix for this will have to be in the client.  The client does not
send protocol to close the prepared statement, but rather reuses the PKGNAMCSN on the PRPSQLSTT
request once the prepared statement has been closed. This is how the server knows to close
the old statement and create a new one.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message