Mailing-List: contact users-help@camel.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@camel.apache.org
Received-SPF: pass (nike.apache.org: domain of claus.ibsen@gmail.com
 designates 209.85.218.228 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type;
        b=gnQFFHlHnsWMwEqZs16JFPB/9vTqU6LCwUH/EDe2yn+ZWCqZcUnt8tSQEzK86iLUgp
         jE9skyz8GcX5vXXe5vJCmofaDo8iC8GOBSufK9dU6/pSuMl8d4H8bWUQbeINWSt2RvDG
         5185Y+xyeVdFGjxYF9XxtOFIdC/2L4nitVEdY=
MIME-Version: 1.0
In-Reply-To: <ab8e556c1002012130w4e96588dk8ae702ae7d6752a7@mail.gmail.com>
References: <27412920.post@talk.nabble.com>
 <ab8e556c1002012130w4e96588dk8ae702ae7d6752a7@mail.gmail.com>
From: Claus Ibsen <claus.ibsen@gmail.com>
Date: Tue, 2 Feb 2010 10:31:52 +0100
Message-ID: <5380c69c1002020131t56d47120w4937bca2ceaaac2e@mail.gmail.com>
Subject: Re: Performance - Camel JPA
To: users@camel.apache.org
Content-Type: text/plain; charset=ISO-8859-1

On Tue, Feb 2, 2010 at 6:30 AM, Kevin Jackson <foamdino@gmail.com> wrote:
> Hi,
> [snip]
>
>> I have ensured that index are put in place for old table and new table.
>> There is no need of second level cache in this scenario. I have used UUID to
>> generate unique key when inserting new record. Yet this apps take 30 mins
>> for 40,000.
>
> Indexes on the new table are going to hurt your insert performance.
> For large data loads, have you tried:
> 1 - push data into a table with no ref integrity (a load table) and no indexes
> 2 - asynchronously (after all the data has been loaded into the load
> table), call a stored procedure that copies the data from load to the
> real table
> 3 - after store proc has run, truncate the load table
>
> Kev
>

Yeah I do not think JPA fits well with ETL kinda work.
http://en.wikipedia.org/wiki/Extract,_transform,_load

There is a zillion other ways to load a lot of data into a database,
and using an ORM will newer be very fast.

Try googling a bit with your database name and ETL etc. And/or talk to
DB specialists in your organization.

If you need to do hand crafted SQL queries you may want to use Spring
JDBC or iBatis etc. Sometimes its just easier to use Spring JDBC as
its a little handy library.

-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus