Mailing-List: contact directory-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List"
 <directory-dev@incubator.apache.org>
From: "Alex Karasulu" <aok123@bellsouth.net>
To: "'Apache Directory Developers List'" <directory-dev@incubator.apache.org>
Subject: RE: [eve] A Buffer Pool (of direct mapped buffers)
Date: Sun, 7 Dec 2003 12:55:12 -0500
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
Thread-Index: AcO85eMXZtC1nYlKQhah1m2R/Vx6AgAAEXLQAAElIrA=
In-Reply-To: <20031207175042.PSLM1942.imf22aec.mail.bellsouth.net@franklin>
Message-Id: <20031207175515.PTOH1942.imf22aec.mail.bellsouth.net@franklin>

Some corrections below to my comments earlier:

> -----Original Message-----
> From: Alex Karasulu [mailto:aok123@bellsouth.net]
> Sent: Sunday, December 07, 2003 12:51 PM
> To: 'Apache Directory Developers List'
> Subject: RE: [eve] A Buffer Pool (of direct mapped buffers)
> 
> > (Sorry for the delayed response, but my home network was having some
> > 'issues'...)
> 
> Np!
> 
> >
> > Wow, I'm glad I didn't respond to quickly...You've definitely given me
> > something to think about. It's a fascinating problem... The idea of
> > pooling a resource between asynchonous listeners, and then reclaiming
> > the resource when everyone is done with it.
> 
> Good synopsis of the problem.
> 
> > I've used refCounting for this in the past.  With the idea that once the
> > ref count of the resource goes to 0, then it can add itself back into
> > the pool.
> >
> > RefCounting is of course, fairly simple to implement, as long as the
> > consumers (client code) are disciplined enough to 'do the right thing'.
> > When you pass a reference to the buffer off to a listener, you can
> > automatically bump its ref count... Then the listener must call
> > release() on the buffer when it's done.
> 
> That's exactly what we're going to have to do if this is the approach we
> take: meaning the use of a buffer pool.  That can mean memory leaks if all
> resources are not released by the stage using the buffer.  This is not
> that big of a deal I think because the components using the buffers with
> not be that many and they will be under our full control. Let's take a
> closer look at the implementation and the mechanics.
> 
> The only things that need to increment the reference count would be:
> 
> 1). the buffer requesting source when the buffer is claimed
> 2). the stage, the event is enqueued onto (the sink), when the buffer is
> enqueued

the Subscriber of the stage is really what needs to be diligent here.  It
needs to register itself as a Buffer Observer perhaps with a BufferPool
service.

> Now the only things that need to decrement the reference count would be:
> 
> 1). the buffer requesting source after the buffer is fired off in an event
> 2). the stage, the event is enqueued onto (the sink), when the buffer is
> dequeued, and processing on the event with the buffer is complete

Let me rephrase #2:

2). the stage asynchronously processing the event with the claimed pool
buffer must release or unregister its listener as an observer of the 
buffer. 

> Also I would not use a reference increment (in an integer) but rather
> an Observer-Observable pattern of sorts where if there are observers on
> the Resource then it is not reclaimed.  Once the Observer count hits
> zero after all Observers have stopped observing the resource then we
> reclaim the buffer.
> 
> So here's what a pooled buffer claim-reclaim cycle might look like:
> 
> 1. Event source claims the buffer
> 2. Event source publishes an event with a read only copy of the buffer
> 3. Event notifier synchronously notifies all subscribers
> 4. Subscribers that asynchronously process the event (by enqueuing the
>    event onto a stage queue) register themselves as observers.
> 5. publish call by the event source returns and the event source
>    releases the buffer ( deregisters itself as an Observer ).
> 6. Other observers continue to asynchronously process the request
>    and release the buffer ( deregister themselves as Observers ).
> 7. Observer count reaches zero the buffer is reclaimed by the buffer
>    pool and will be reused.
> 
> How's that sound Mark? Anyone else care to comment as well?
> 
> Alex
>