openjpa-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Pflueger <Michael.Pflue...@sma.de>
Subject AW: Speed of fetching simple entities using OpenJPA
Date Tue, 13 Sep 2011 14:33:28 GMT
I think this test also demonstrates that PostgreSQL Cursors aren't working with OpenJPA.

When I fetch 5 million entries using JDBC and iterate over them with cursors, heap memory
consumption stays below 4 MB.

When I fetch 2 million entries using OpenJPA with FetchBatchSize=2000, heap memory increases
to 125MB.
When I do it without FetchBatchSize set, memory usage is even higher, it hits 100% heap memory
usage at 250MB.

So, what I think might happen is that with FetchBatchSIze set, OpenJPA streams through the
results of the JDBC driver, but the driver itself loads all result rows into memory at once
and does not use cursors.

With FetchBatchSize unset, OpenJPA tries to instantiate all entities at once and thus memory
usage is even higher.

So, something is probably wrong in the communication between the OpenJPA and the Postgre driver?

Michael

-----Urspr√ľngliche Nachricht-----
Von: Michael Pflueger [mailto:Michael.Pflueger@sma.de] 
Gesendet: Dienstag, 13. September 2011 15:29
An: users@openjpa.apache.org
Betreff: AW: Speed of fetching simple entities using OpenJPA

Hi,
attached is a zip containing both the OpenJPA and JDBC tests to read the entities, and a small
prog to fill the DB. I'm doing entytymanager.clear() every 2000 records so memory does not
fill up, and have tried to do it only every 20k records, but that seemed to make it a bit
slower, not faster.

Michael
___________________________________________________

SMA Solar Technology AG
Aufsichtsrat: Guenther Cramer (Vorsitzender)
Vorstand: Juergen Dolle, Roland Grebe, Uwe Hertel, Pierre-Pascal Urbon, Marko Werner
Handelsregister: Amtsgericht Kassel HRB 3972
Sitz der Gesellschaft: 34266 Niestetal
USt-ID-Nr. DE 113 08 59 54
WEEE-Reg.-Nr. DE 95881150
___________________________________________________


Mime
View raw message