db-torque-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ludwig Magnusson" <lud...@itcatapult.com>
Subject RE: Extensive memory usage in torque-generator. Bug?
Date Tue, 24 Nov 2009 20:21:07 GMT
In reply to this, I actually found the maven option to extend the amount of
memory.
Now the VM can use 512Mb. I tried with my _actual_ data-xml file (which was
created by torque:datadump) and that task succeeded but used 415Mb. My
computer has 4Gb of RAM but extending the allocated memory does not seem
like a solution. A database can grow much larger than the one we have.
We will take a look at the generator code and see if there is anything we
can find. (that would be the first place to look at right?)
/Ludwig



-----Original Message-----
From: Thomas Fischer [mailto:fischer@seitenbau.net] 
Sent: den 24 november 2009 21:09
To: Apache Torque Developers List
Subject: RE: Extensive memory usage in torque-generator. Bug?

Hm, on the one hand 65 MB RAM is not much on a modern computer. On the
other hand, it is much as compared to the data, as you said.
I am afraid that probably we do not have the manpower to dig into this;
memory usage is difficult to track down. If you find a reason for the
extensive memory usage and propose a solution, We'd be glad to incorporate
it.

For your immediate problem, try increasing the maven memory limits. (the
usual -Xmx -Xms options for the java vm).

     Thomas


> A few days ago I posted in the users list that I got a
> java.lang.OutOfMemoryError while trying to generate my data-sql.
>
> I did some stress testing on the generator, here is how it went:
>
>
>
> I used a simple table and tried to insert different number of rows,
ranging
> from 1 to 2000.
>
> This is my data.xml for n number of roles:
>
> <dataset name="all">
>
>   <Role Id="0" Name="Role_0"/>
>
>   <Role Id="1" Name="Role_1"/>
>
>   .
>
>   <Role Id="n" Name="Role_n"/>
>
> </dataset>
>
>
>
> At the end of each run maven prints the amount of memory used (final
> memory), here are some values:
>
>
>
> 1 row - 11M/20M
>
> 100 rows - 15M/25M
>
> 1000 rows - 42M/63M
>
> 1500 rows - 58M/63M
>
> 2000 rows - fail 63M/63M
>
>
>
> The limit seems to be around 1500 rows.
>
> When that's the case, the input file is ca 60Kb and the output file is ca
> 100Kb, so how can the generator use 58Mb of memory?
>
> Is this a bug? Known or unknown?
>
> /Ludwig
>


---------------------------------------------------------------------
To unsubscribe, e-mail: torque-dev-unsubscribe@db.apache.org
For additional commands, e-mail: torque-dev-help@db.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: torque-dev-unsubscribe@db.apache.org
For additional commands, e-mail: torque-dev-help@db.apache.org


Mime
View raw message