db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Matrigali <mikem_...@sbcglobal.net>
Subject Re: [Fwd: Controlling file handle usage]
Date Wed, 20 Jun 2007 18:11:53 GMT
off hand I don't know if there exists a derby entry, we have talked 
about things like this off and on, ie. providing resource control on:
memory, open files, disk, ....  We kept hoping that java/jvm would
provide features to help in this area.

There exists an undocumented property to control the number of files
that the store cache's open, which was used in the past to allow the
system to run where open files were severely restricted.  I think it
defaults to 100 now and in past I think we tested with 20.  But I 
believe some changes have gone in that allow for multiple opens to
avoid concurrency problems so not sure this even does what it used to
(thus it is undocumented).  Also this is just one specific part of the
code and does not interact with all the other users of file pointers.

I think to implement correctly one would want a single module that
monitored everyone of this resource.   The current cache code might
work, but all code would have to be tied in rather than many different
instances.  Many uses are just necessary and can't be scaled back.  Off
hand I can think of the following file resources on server side probably 
a different set on client side - and since this is
a system resource I think java is not really going to tell you how the
java action maps to the system dependent resources:
     o store caches a number of open database files, defaults to 100,
       see derby.storage.fileCacheSize in BaseDataFileFactory.java
     o the log will open and close files as each new log file comes 
about, probably needs to have at least 2 open during switch.
     o the sorter is currently very greedy on open files when it does a
       disk based sort.  I think there is a derby for allowing it to
       open/close files.
     o need an open file for the error log, i think we keep it open.
     o in client/server we are going to have open files to do 
communication (this is probably OS dependent), and the actual number
depends on jdbc configuration for shared channels.
     o Not sure about the new lob/clob locator stuff it may be 
maintaining separate cache of open files.

Of all these I think currently the store is the only one set up to
handle cutting back on open files.  I think the sorter work would not
be too hard (the hard part is figuring out when to do it - not the
actual mechanism).

Bryan Pendleton wrote:
> Didn't get any response on the users list, so I'm trying the dev list.
> If this capability doesn't currently exist, should I file
> an improvement suggestion?
> Does anybody have any initial reactions as to how something
> like this might be built?
> thanks,
> bryan
> ------------------------------------------------------------------------
> Subject:
> Controlling file handle usage
> From:
> Bryan Pendleton <bpendleton@amberpoint.com>
> Date:
> Mon, 18 Jun 2007 07:42:45 -0700
> To:
> Derby Discussion <derby-user@db.apache.org>
> To:
> Derby Discussion <derby-user@db.apache.org>
> Hi,
> Is there a configuration setting which I can use to
> set a limit on the total number of file handles which
> Derby will use?
> I'm trying to run Derby inside of various application server
> environments, many of which have limited resources available
> for each application, and I'd like to configure Derby to
> "play nice" and confine itself to only a small portion of
> the overall server's resources.
> Any suggestions about how to do this?
> thanks,
> bryan

View raw message