I am running a system that is opening a fairly large number of databases within a single server environment. The problem is each database is keeping open a certain number of files, and on some systems is causing failures from too many open files.
I am trying to work around this by shutting down the database whenever it's not in use, but my current system regularly scans all databases every five seconds, and so I'm booting and shutting down the databases constantly. This is exacerbated by the fact that I have other threads that are also updating some subset of these databases.
I have tried to deal with this through a central registry that tracks usages of a database and shuts it down only when it's not in use, but it's been difficult to get right, and I either end up with leaks or with exceptions caused by one thread being in the middle of updating a table while another thread is shutting it down. The different kinds of exceptions I get are quite varied and nasty, and involve not only errors caused by shutdown, but recovery errors when a database was shut down while it was also being updated.
So, I have a couple of questions for you all:
- Do the number of files held open by a database grow as the number of rows in a table grows? Some of these tables will have millions of records.
- Is there any way to tell Derby to close out open files and/or reduce the number of open files to a minimum?
- Is it a known issue that shutting down a database will corrupt the database if another thread is updating it?
David W. Van Couvering