Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Derby Development" <derby-dev@db.apache.org>
Received-SPF: neutral (hermes.apache.org: local policy)
Message-ID: <42544BF0.4020307@debrunners.com>
Date: Wed, 06 Apr 2005 13:52:00 -0700
From: Daniel John Debrunner <djd@debrunners.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
 rv:1.7.3) Gecko/20040910
MIME-Version: 1.0
To: Derby Development <derby-dev@db.apache.org>
Subject: Re: more background threads
References: <424AA335.1090405@sun.com> <424C5BC6.10900@sbcglobal.net>
 <425445D4.1030003@sun.com>
In-Reply-To: <425445D4.1030003@sun.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

David Van Couvering wrote:

> OK, I think I'm beginning to get my head around the background thread
> design.  It seems to be used in a lightweight way by a number of various
> components, but is most heavily used by the transaction manager to
> handle post-commit work.
> As you mentioned, the current design spreads the work across the user
> threads if the background thread gets backed up, and I agree that
> overall throughput may not be much improved by having multiple
> background threads do the work. 

The intention of the item on the to-do list was to reduce the number of
threads not increase them :-). The issue is that if a Derby system is
booted with 1,000 databases, then 1,000 backgound threads are created,
all just to handle background work. The to-list item was intended change
the DaemonFactory from being booted per-database to being booted
per-system.  Thus a single backgound thread would handle the idle
requests. Then, I agree, the DaemonFactory could be enhanced to support
a handful of worker threads behind the interface, without the knowledge
of the caller.

The work involved I think is more involved than just changing how the
factory is booted, since at the moment the context is tied to the
thread, the change would require the context be separated from the
thread and set dynamically as work came in from each database.
In the end there would be a DaemonFactory per system and a DaemonService
per database.


> While reading the code, I did encounter one thing that had me
> concerned.  What I am calling the transaction manager
> (impl.store.Xact.java) seems to me to be overstepping its
> responsibilities:  it takes care of running work items when the work
> item requires immediate attention or if the daemon thread is backed up. 
> This logic is not made available to any of the other components that
> make use of the daemon service.  One can imagine this logic being
> refactored out and made generally available to all clients of the daemon
> service.  Is there a specific reason for having this code in Xact.java?

I think it was a quick change to address a customer specific issue. I
agree that cleanup is needed here, I once started some code that tried
to cleanup the Serviceable intefaces, so that there was one set of
constants (something like DONE,REQUEUE,IMMEDIATE, USERSPACE) and thus
similar state is not separated over the return values from
performWork(), serviceASAP and serviceImmediately() and the value passed
into the enqueue method).
Currently it is not well defined what should happen if the serviceNow
parameter passed to enqueue is in conflict with the returns from
serviceNow or serviceImmediately(). You can see that some of the
implementations of Serviceable implement the current methods
inconsistently (from memory).


Thus I imagined a single method on Serviceable that indicated how the
operation wanted to be serviced, make performWork void and don't have a
serviceNow on enqueue.


Dan.