cayenne-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephane Claret (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (CAY-1225) Controlling JDBC Driver setting "fetchSize" in a SelectQuery
Date Wed, 20 May 2009 13:16:45 GMT

    [ https://issues.apache.org/jira/browse/CAY-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711131#action_12711131
] 

Stephane Claret edited comment on CAY-1225 at 5/20/09 6:15 AM:
---------------------------------------------------------------

Modified files:
Original come from v3 M6 generic-unpublished version.

-People who can't wait for this feature to be publicly available can use this patch. Modified
sections are marked with comment "quick fix".

-I added cursorFetchSize property in SelectQuery (nullable Integer). If set, it is used to
modify the preparedStatement in SelectAction.performAction otherwise it has no effect, there
should be no risk to break existing code.

The same fix can be applied to current v2 sources (2.04) too, but in that case, if you're
using ResultIterator and convert DataRows to DataObjects, you still have to manually unregister
them from the DataContext in your iteration loop in order to avoid memory issues.

      was (Author: stefcl):
    Modified files:
Original come from v3 M6 generic-unpulished version.

-People who can't wait for this feature to be publicly available can use this patch. Modified
sections are marked with comment "quick fix".

-I added cursorFetchSize property in SelectQuery (nullable Integer). If set, it is used to
modify the preparedStatement in SelectAction.performAction otherwise it has no effect, there
should be no risk to break existing code.
  
> Controlling JDBC Driver setting "fetchSize" in a SelectQuery
> ------------------------------------------------------------
>
>                 Key: CAY-1225
>                 URL: https://issues.apache.org/jira/browse/CAY-1225
>             Project: Cayenne
>          Issue Type: Improvement
>          Components: Cayenne Core Library
>    Affects Versions: 2.0 branch
>            Reporter: Stephane Claret
>            Priority: Critical
>         Attachments: SelectAction.java, SelectQuery.java
>
>
> ResultIterator is a nice API for dealing with large SELECT queries, unfortunately some
JDBC drivers that don't automatically use server-side cursors (eg PostgreSQL JDBC driver)
wont'give access to the ResultSet (at JDBC level) until the whole query has finished executing
and all rows have been returned.
> It basically mean that if you execute a SelectQuery returning 1'500'000 rows, there's
a high chance of getting a OutOfMemoryException in the JDBC driver code (Statement.Excecute),
before Cayenne even gets a chance to call the ResultSet.Next().
>  
> It makes the ResultIterator API much less efficient than it could be because :
> -Even if rows are processed one by one, they need to be stored in local memory, causing
heap size issues.
> -In some scenarios, user code could start working with the resulting rows immediately
as they become available, while the select query is still being executed by the database engine.

> To solve this problem, there should be a way to have the SelectQuery object calling Statement.setFetchSize()
before it executes.
> A SelectQuery.setFetchSize( int ) method would be perfect because it's often needed to
configure this setting on a per-query basis.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message