db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <de...@segel.com>
Subject Re: order by
Date Thu, 02 Mar 2006 16:32:09 GMT
On Thursday 02 March 2006 5:00 am, Thomas Vatter wrote:

This doesn't make any sense.
You keep saying "spreadsheet".

Are you reading directly from an excel spreadsheet? .XLS or did you convert 
the spreadsheet to a .CSV format?

The other issue of concern is that you say that you continue to die at 
approximately 1200 rows. 

The first thought is that its a data set problem because you indicate that it 
occurs with a single data set.

The second thought is that its a programming issue on your part because you 
now say that you're using different portions of the data set and its still 
hanging at 1200 rows.

A simple read/parse/insert program should not eat memory.

Are you using a prepared statement?

As you parse each line of data, how are you storing the fields and when you do 
your insert, are you properly "flushing" the variables? (Destroying the 
objects...)


The only negative thing is that you can't use the split() method. You have to 
manually parse the string to pull out the data fields, strip the quotes and 
white space.... 
(Which is why ESQL/C  or C is a better choice for this type of work...)


HTH,

-Mike

> Kristian Waagan <Kristian.Waagan@...> writes:
>
> ...
>
>
> Hello Kristian,
>
> I'm doing this on a Windows XP system, I start all java processes with
> -Xmx1024m, I had a try with 2048 but the process did not start so I stayed
> with 1024. "Between 912 and 960" is the total usage for the machine,
> read from the task-manager. I have thought about it, derbys pageSize
> is 4k, I have increased pageCacheSize from 1.000 to 10.000, resulting
> in 40mb steps, this would explain that memory "ends" before 1024 is
> reached. Yes, I was unprecise for shortness with the word "record".
> I should have said "record in the main database table" or "dataset".
> But you are right, a dataset (or line) in my spreadsheet has around
> 30 fields only. I had the network server running on the described machine,
> because I am testing how derby performes in the network situation,
> because in the single user situation I have typically less data.
>
> tom

-- 
--
Michael Segel
Principal 
Michael Segel Consulting Corp.
derby@segel.com
(312) 952-8175 [mobile]

Mime
View raw message