camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: Performance - Camel JPA
Date Tue, 02 Feb 2010 09:31:52 GMT
On Tue, Feb 2, 2010 at 6:30 AM, Kevin Jackson <foamdino@gmail.com> wrote:
> Hi,
> [snip]
>
>> I have ensured that index are put in place for old table and new table.
>> There is no need of second level cache in this scenario. I have used UUID to
>> generate unique key when inserting new record. Yet this apps take 30 mins
>> for 40,000.
>
> Indexes on the new table are going to hurt your insert performance.
> For large data loads, have you tried:
> 1 - push data into a table with no ref integrity (a load table) and no indexes
> 2 - asynchronously (after all the data has been loaded into the load
> table), call a stored procedure that copies the data from load to the
> real table
> 3 - after store proc has run, truncate the load table
>
> Kev
>

Yeah I do not think JPA fits well with ETL kinda work.
http://en.wikipedia.org/wiki/Extract,_transform,_load

There is a zillion other ways to load a lot of data into a database,
and using an ORM will newer be very fast.

Try googling a bit with your database name and ETL etc. And/or talk to
DB specialists in your organization.

If you need to do hand crafted SQL queries you may want to use Spring
JDBC or iBatis etc. Sometimes its just easier to use Spring JDBC as
its a little handy library.

-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Mime
View raw message