db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dyre.Tjeldv...@Sun.COM
Subject Re: Reducing memory usage
Date Sun, 07 Oct 2007 14:58:13 GMT
Charlie Hubbard <charlie.hubbard@gmail.com> writes:

> Hi,
> I have a database that I have been storing binary data in a Blob
> column.  I've decided that it was a bad idea and I'd like to remove it
> from the DB and store it on the filesystem.  I've written a small
> program to extract the contents from the DB and save it to the file
> system.  It works except it runs out of memory using the default 64
> MB.  While I could increase the -Xmx setting it seems superflous.
> I've profiled the application and majority of the memory is being
> occupied by derby.  I see lots of classes like
> org.apache.derby.impl.services.cache.CachedItem and
> org.apache.derby.impl.store.raw.data.StoredPage holding 32KB byte
> buffers.
> Is there a way to change derby so that it would drop those cached
> buffers?  

The size of the page cache is controlled by the
derby.storage.PageCacheSize property. Its default value is 1000, so
you could try to reduce it. Normally you need to adjust your -Xmx
setting so that you have enough heap for the page cache + some
overhead for the rest of Derby...

Reducing it could affect performance but that depends on the nature of
your application (how much it benefits from caching...)

>Or a way to turn off caching for blobs? 

Don't think so

> Or is there a way
> to optimize the memory a little more so that it would run in a smaller
> footprint?

Well, the page cache is the part of Derby that normally uses the most
memory, so it's probably a good place to start.

> This program will have to run on DB that could be in the GB size.  I
> just need to extract the contents and save it out to disk.  

If you only read each record once before writing it to disk, you
probably won't benfit much from caching anyway, so reducing it is
probably a good idea...

I think there is a lower limt for the page cache size, but I don't
remember what it is. The tuning guide has the details.


View raw message