commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Osipov (JIRA)" <>
Subject [jira] [Created] (DBUTILS-126) Add iterable interface approach
Date Wed, 12 Aug 2015 18:49:45 GMT
Michael Osipov created DBUTILS-126:

             Summary: Add iterable interface approach
                 Key: DBUTILS-126
             Project: Commons DbUtils
          Issue Type: New Feature
    Affects Versions: 1.5
            Reporter: Michael Osipov

Currently, DbUtils allows to load all data at once into memory. In many cases, when dealing
with 100 000+ records, this will cause an OOME. I have to iterate over 100 000 lines with
10 columns and retrieve subdata which are in turn indexed with Lucene.

Because I cannot iterate over the entire dataset without memory problems, I have to retrieve
them pagewise. (fiddling with rownums in Oracle).

An iterator interface which streams this down from the DB to the client would solve such a
problem, very much like [Spring JdbcTemplate extension that returns iterators|].
This works very well in a Spring app downloading a table with two million records directly
to the HTTP client without any buffers.

This message was sent by Atlassian JIRA

View raw message