hivemind-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Carman" <ja...@carmanconsulting.com>
Subject RE: Potential memory leak with Threaded model and Events listeners
Date Sun, 05 Dec 2004 13:38:35 GMT
I personally don't even like the automatic event listener registration
facility provided by BuilderFactory, but maybe that's just me.  Anyway, what
if people actually do want each created instance (in a pool) to receive the
event notification?  Do we force those folks to do it manually (as you
explained in your workaround)?  Which case do you think would be the
majority (I would have no idea, really)?

If I had time to write it, I'd try to come up with an event dispatching
facility which was tightly integrated with HiveMind's lifecycle management
(knows about discard, passivate, and activate).  First, I'd create a
mechanism where you could register for lifecycle events in HiveMind...

// A new service in HiveMind...
public interface LifecycleManager 
{
  public void addLifecycleListener( LifecycleListener l );
  public void removeLifecycleListener( LifecycleListener l );
}

public interface LifecycleListener
{
  public void objectPassivated( Object o );
  public void objectActivated( Object o );
  public void objectDiscarded( Object o );
  // public void objectCreated( Object o ); ?
}

Maybe anything implementing LifecycleListener would automatically be
registered, but then again, maybe not.  Anyway, then I'd create an event
dispatcher service...

public interface EventDispatcher
{
  public void registerEventListener( String topicId, Object listener );
  public void deregisterEventListener( String topicId, Object listener ); 
  public Object createListenerProxy( String topicId, Class listenerInterface
);
  public Object createListenerProxy( String topicId, Class
listenerInterface, boolean disregardPassivatedInstances );
}

I chose to call them "topics" because I am loosely modeling this idea on the
notion of JMS topics.  The implementation of MessageDispatcher would
implement the LifecycleListener interface and be registered with the
LifecycleManager.  Here's how you would use it manually...

public class MyServiceImpl implements MyService
{
  private MyEventListener listenerProxy;
  private EventDispatcher dispatcher;  // To be injected!
  public void init()
  {
    listenerProxy = ( MyEventListener )dispatcher.createListenerProxy(
"MyService", MyEventListener.class, true );
  }

  public void someMethod()
  {
    listenerProxy.someMethodInvoked( new MyEvent( this ) );
  }
}

public class MyOtherServiceImpl implements MyOtherService, MyEventListener
{
  private EventDispatcher dispatcher;
  public void init()
  {
    dispatcher.registerEventListener( "MyService", this );
  }

  public void someMethodInvoked( MyEvent e )
  {
  }
}

Here, the implementation of MyService has decided that it wants to be able
to "publish" events to the "MyService" topic via the MyEventListener
interface (note, multiple listener interfaces could be used on the same
topic).  So, it asks the EventDispatcher to create a proxy object which
implements the listener interface (and also skips over passivated objects).
When it wants to "publish" an event, all it has to do is call a method on
the proxy object and all registered listeners will receive the event
notification!  On the other side of things, the implementation of
MyOtherService has decided that it would like to listen for events via the
MyEventListener interface on the "MyService" topic.  Of course, I can see
easily how we could extend the hivemodule.xml file to facilitate this
automagically...

<set-listener-proxy property="listenerProxy" topicId="MyService"
listenerInterface="com.myco.MyEventListener" skipPassivated="true" />

<register-listener topicId="MyService"  />

These would be new additions of BuilderFactory's schema for creating
listener proxies.  One cool thing about this is that you don't have to
register to receive events on each and every source of the event.  As a
matter of fact, you don't even necessarily have to know where they're coming
from.  All you need to know is the topicId and implement the appropriate
listener interface to receive events!  I might want to add in a new method
for registering an event listener where you can specify exactly which
listener interfaces you wish to use to receive events on the specified
topic, but that may be overkill for most situations.  We would also have to
agree on what the default behavior is with respect to skipping passivated
listeners or always make the user specify what they want.  Anyway, so much
for checking my email quickly and heading out the door!  Hope I didn't use
up too much of your allotted email space!




-----Original Message-----
From: Jean-Francois Poilpret [mailto:jfpoilpret@hcm.vnn.vn] 
Sent: Saturday, December 04, 2004 7:34 PM
To: hivemind-user@jakarta.apache.org
Subject: RE: Potential memory leak with Threaded model and Events listeners

No, I don't want each __created__ instance to receive notifications, I just
want each __active__ instance to receive them, and any __inactive___
instance not to receive them.
Currently with either threaded or pooled model (as far as I can see), all
created instances actually receive notifications, even when inactive. Worse,
for threaded model, inactive instances never get garbage-collected because
they are strongly referenced by the service which fire events.

For me, the expected behavior of HiveMind would be:
- when a service instance gets created (threaded) or activated (pooled),
then it should be added as a listener to the service firing events
- when a service instance gets discarded (threaded) or deactivated (pooled),
then it should be removed as a listener from the service supplying events.

NB: this description applies to the case where the event-firing service is a
singleton (my situation right now), but the behavior would/should probably
be different if that service used a different model.

Currently, the best work-around is:
- for threaded model, the listening service must implement Discardable and
remove itself from the listeners list of the event firing service, when its
threadDidDiscardService() method is called. The registration as a listener
can either be done by configuration (BuilderFactory) or in the constructor.

- for pooled model, the listening service must implement PoolManageable and
it must register itself as a listener in its XXXX methods, and unregister
itself when its XXXX() method is called. Here automatic Event registration
through BuilderFactory is useless.

These work-arounds work well, but they are not very "clean" in the sense
that this is code the end-developer has to write that HiveMind could
directly handle for him.
In addition, in the threaded situation, if we use the BuilderFactory to wire
the listener to the event supplier, then the work-around introduces some
asymmetry (registration in config, unregistration in the code), making the
use of BuilderFactory for this aspect useless.

I think before changing anything in HiveMind, we (all concerned developers)
should discuss the various situations (models for supplier/consumer:
singleton/threaded, singleton/pooled, threaded/singleton,
threaded/threaded...) and specify for each what the behavior should be, then
the actual coding can start. Should we start a new page on the WIKI?

Cheers

-----Original Message-----
From: Howard Lewis Ship [mailto:hlship@gmail.com] 
Sent: Sunday, December 05, 2004 4:09 AM
To: hivemind-user@jakarta.apache.org
Subject: Re: Potential memory leak with Threaded model and Events listeners

If you want each created instance to receive the notification, use the
pooled, not threaded, service model.  Pooled instances exist
indefinately once created.  The only down side is that code related to
the event needs to be thread safe, and that deactivated instances in
the pool will also receive notifications.


On Sun, 5 Dec 2004 00:18:27 +0700, Jean-Francois Poilpret
<jfpoilpret@hcm.vnn.vn> wrote:
> Hi Luke,
> 
> Thank you for your remark.
> Although your remark is correct in many cases of Listeners, I think it
would
> generally not apply in HiveMind context, I mean (if people follow the
"right
> path") that normally, only a service will implement a Listener interface,
> thus at least HiveMind itself (through one of the generated proxies) will
> hold a strong reference to that service/listener. But of course, if
someone
> decides to create a Listener on-the-fly and add it "by hand" to the event
> supplier service, then the problem you described would occur.
> 
> Anyway, I just did some tests (with strong and with weak references) and
in
> fact, the result is not "as good" as I expected initially. Indeed, when
> suggesting that idea, I simply forgot that the handling of weak references
> is totally non-deterministic, ie the weak ref could still be valid even
> though there is no more existing strong ref. The time at which the weak
ref
> would be cleaned is totally up to the GC...
> So as my test have shown me, using weak refs improve the issue, but they
do
> not solve it really:-(
> 
> In my tests, I did something like that:
> - create EventSupplier service (singleton)
> - create EventConsumer service (threaded)
> - instantiate EventConsumer in 1000 "threads" (they are real threads but
> they do not run concurrently, but I force consecutive runs, with a call to
> Registry.cleanupThread() at the end of each thread)
> - each thread call the EventSupplier to generate a new event
> - all calls are traced
> - each EventConsumer instance allocates 10KB on the heap and never release
> them.
> 
> With strong refs:
> - the 1000th thread sends an event that is received by 1000 EventConsumer
> instances
> 
> With weak refs:
> - the 1000th thread sends an event that is received by about 30
> EventConsumer instances.
> 
> Of course that is much better, but the ideal, here in my testcase, would
be
> always just one EventConsumer instance receiving the event.
> 
> So finally, maybe I will stick with the previous -not very clean from
design
> viewpoint- workaround:
> - have EventConsumer implement Discardable and remove itself from the
> EventSupplier list of listeners, when threadDidDiscardService() is called.
> 
> And then I will eagerly wait for HM1.1 release (hope it will fix this
> issue).
> 
> Cheers
> 
> 
> 
> -----Original Message-----
> From: Luke Blanshard [mailto:luke@blanshard.us]
> Sent: Saturday, December 04, 2004 8:13 PM
> To: hivemind-user@jakarta.apache.org
> Subject: Re: Potential memory leak with Threaded model and Events
listeners
> 
> Jean-Francois,
> 
> The problem with holding listeners weakly is that people commonly create
> listener objects on the fly and then retain no other references to them
> once they've added them.  So these listeners are thrown away on the next
> GC cycle, and they stop working.
> 
> One solution to this is to have a tagging interface (we call ours
> WeakListener) that tells the event source that it should hold the
> listener weakly.  All listeners that don't implement this interface are
> held strongly.  This way, you as the implementer of the listener are
> telling the event source whether to hold the listener weakly (meaning
> you better have a strong reference to it) or strongly (meaning you don't
> have to).  As you say, it's pretty easy to write a listeners collection
> that honors this WeakListener interface and throws away the expired
> listeners when you ask for the current list of listeners.  (We don't use
> an iterator, but rather a method that returns an array of listeners, for
> thread safety reasons.)
> 
> Luke
> 
> Jean-Francois Poilpret wrote:
> 
> >...
> >To Howard:
> >During the night (French people say that the "night gives good
advice";-)),
> >I thought of a quite easy workaround, that can be done at the event
> >supplying service level: just use a weak reference to the listeners
> (instead
> >of a strong one), I think that should do the trick.
> >Maybe it would even be possible to write a general "WeakEventListeners"
> >class to manage all that for the event supplier:
> >- when addListener, is called generate a weak reference to the added
> >listener and store it in the list
> >- create a specific Iterator for WeakEventListeners that would skip the
> refs
> >that have become weak-reachable (and remove these from its list at the
same
> >time).
> >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
> 
> 


-- 
Howard M. Lewis Ship
Independent J2EE / Open-Source Java Consultant
Creator, Jakarta Tapestry
Creator, Jakarta HiveMind
http://howardlewisship.com

---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org


Mime
View raw message