db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristian Waagan <Kristian.Waa...@Sun.COM>
Subject Re: Inserting data into a database on a Derby network server
Date Mon, 08 Oct 2007 11:12:31 GMT
Ture Munter wrote:
> I'm using embedded derby in a Java SE application, however I use Derby
> as a network server while building the database. In that connection I
> ran into problems.
> I have a database with 65000 rows that I wanted to add to a Derby
> database (total size before insert around 10000 records, 30MB). Each row
> consist of some strings and some floats (6 fields in total) and the
> operation requires two inserts and one select. During insert I run Derby
> as a Derby network server, the problem is that no matter what I do I get
> OutOfMemory exceptions (Derby ran out of heap space) all the time.
> Setting the max memory use of the JVM to 1GB allowed me to add around
> 17000 rows, but not anymore than that.

Hello Ture,

Is this a out-of-the-box configuration, or have you configured Derby by 
specifying properties?
If you have tuned it, have you set the Derby page cache size? 
(derby.system.pageCacheSize).
The page size?
Is there only one connection inserting data?

Sounds to me as if you have a program generating the data. Is this 
something you could share with the community?
That would be great, then people could run it and see if they see the 
same behavior. Also, if there's a bug it will be easier to find it if we 
have a running repro.

If that is not possible, can you run with Java SE 6 and tell it to dump 
the heap on OOME?
Another possibility is to use jmap to get a histogram of the 
heap-content (add the option to only get live objects as well), and this 
would be easy to post.

> 
> The problem was "solved" by running Derby embedded in the program that
> inserted data into the database and only commit every 50th row. When
> running Derby embedded I didn't run out of memory. After adding all
> 65000 records I can still run Derby as network server and do queries,
> but there are problems when trying to insert data. There are no problems
> when running Derby embedded. Have anybody experienced the same
> problem/features?
> 
> I also experience a decrease in performance after having inserted many
> rows, but the slow-down is not as bad as described in the thread
> "exponential increase in insert time".

Just out of curiosity, are you using Statement.executeUpdate, prepared 
statements or batching when inserting the data?

> 
> Apart of the problems when inserting data in bulk, embedded Derby works
> great in my application.

That is nice to hear :)


regards,
-- 
Kristian


> 
> Thanks
> - Ture
> 


Mime
View raw message