geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jay D. McHugh" <...@joyfulnoisewebdesign.com>
Subject Re: [discuss] Release Geronimo 1.2
Date Sat, 07 Apr 2007 02:31:02 GMT
Hello all,

New attempt that doesn't just hide the problem.

I managed to find reference on how to make a linked list behave as 
synchronized (above and beyond simply trying to access them from within 
synchronized code blocks).

Attached is the patch - It is actually for OpenEJB...I'm going to wait 
until someone has a chance to confirm that it actually resolves the 
Geronimo/Daytrader issue then make the JIRA over on OpenEJB.

Jay

David Jencks wrote:
> I don't think this is acceptable.  There should be only one thread 
> working with the context at a time.  Either this exception is caused 
> by modifying the collection in the same thread in which case we can 
> fix it easily or it is caused by more than one thread having access to 
> a context at once.  Since by my reading of the code this is a context 
> attached to a stateless session bean instance, that would mean that 
> more than one thread is using a stateless session bean instance at 
> once, which is definitely cause to -1 the release.
>
> I hope there's another possibility I haven't thought of..... but 
> hiding the problem is not acceptable unless we really understand what 
> is going on and are really convinced it's harmless.
>
> thanks
> david jencks
>
> On Apr 6, 2007, at 1:40 PM, Jay D. McHugh wrote:
>
>> Chris (do you go by Chris or Christopher?),
>>
>> Here is a patch that I just wrote that allows the exit routine of 
>> ConnectionTrackingCoordinator to finish cleanly after a number (5) of 
>> attempts at removing the resource.
>>
>> If it fails after five tries, then the routine exits and throws a 
>> ResourceException (that will hopefully be caught further up the stack).
>>
>> Would you like to try it to see if it solves your concurrency problem?
>>
>> Jay
>>
>> Christopher Blythe wrote:
>>> Doubtful... everything tested fine under light browser based 
>>> testing. As the exception suggests, this is a concurrency problem 
>>> that you would only hit under load.
>>>
>>> On 4/6/07, * Jay D. McHugh* <jay@joyfulnoisewebdesign.com 
>>> <mailto:jay@joyfulnoisewebdesign.com>> wrote:
>>>
>>>     If Matt had no problem deploying and testing DT, could it be a Java
>>>     version or classpath issue?
>>>
>>>     That could explain the difference in the exception during 
>>> deployment
>>>     (and the problems during deployment could possibly explain the run
>>>     time
>>>     problems).
>>>
>>>     Jay
>>>
>>>     Christopher Blythe wrote:
>>>     > I use a commercial load driving tool... FYI, I'm fairly 
>>> certain that
>>>     > G-2.0 has the same issue.
>>>     >
>>>     > On 4/6/07, *David Jencks* < david_jencks@yahoo.com
>>>     <mailto:david_jencks@yahoo.com>
>>>     > <mailto:david_jencks@yahoo.com <mailto:david_jencks@yahoo.com>>>
>>>     wrote:
>>>     >
>>>     >     I think we need to figure out why the
>>>     >     concurrentModificationException is happening before we
>>>     release.  I
>>>     >     think that one possible reason is that we are multithreading
>>>     >     stateless session bean instances.  I hope this isn't the
>>>     cause....
>>>     >     but IMO we need to find out.
>>>     >
>>>     >     Chris, how do you run the several clients?  manually or with
>>>     a tool?
>>>     >
>>>     >     thanks
>>>     >     david jencks
>>>     >
>>>     >
>>>     >     On Apr 6, 2007, at 11:09 AM, Christopher Blythe wrote:
>>>     >
>>>     >>     Gave it a shot... no luck. As soon as I started 2 
>>> clients, the
>>>     >>     same exceptions started to pile up. I have attached the
>>>     >>     geronimo.log. Also, noticed the following exception during
>>>     startup.
>>>     >>
>>>     >>     14:05:00,640 ERROR [TransportConnector] Could not accept
>>>     >>     connection from /127.0.0.1:28428: java.io.IOException: Wire
>>>     >>     format negociation timeout: peer did not send his wire 
>>> format.
>>>     >>     java.io.IOException: Wire format negociation timeout: 
>>> peer did
>>>     >>     not send his wire format.
>>>     >>         at
>>>     org.apache.activemq.transport.WireFormatNegotiator.oneway
>>>     >>     (WireFormatNegotiator.java :88)
>>>     >>         at
>>>     >>        
>>> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:47) 
>>>
>>>     >>         at 
>>> org.apache.activemq.broker.TransportConnection.dispatch
>>>     >>     (TransportConnection.java :1138)
>>>     >>         at
>>>     >>        
>>> org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:805)

>>>
>>>     >>         at
>>>     >>     org.apache.activemq.broker.TransportConnection.start
>>>     (TransportConnection.java
>>>     >>     :885)
>>>     >>         at 
>>> org.apache.activemq.broker.TransportConnector$1.onAccept
>>>     >>     (TransportConnector.java:148)
>>>     >>         at
>>>     >>     org.apache.activemq.transport.tcp.TcpTransportServer.run
>>>     (TcpTransportServer.java:167)
>>>     >>         at java.lang.Thread.run (Thread.java:797)
>>>     >>
>>>     >>
>>>     >>
>>>     >>     On 4/6/07, *Matt Hogstrom* < matt@hogstrom.org
>>>     <mailto:matt@hogstrom.org>
>>>     >>     <mailto:matt@hogstrom.org <mailto:matt@hogstrom.org>>>

>>> wrote:
>>>     >>
>>>     >>         Only a very light load from a few browsers.  One thing
>>>     to try
>>>     >>         is to increase the number of SLSBs in the pool.
>>>     >>
>>>     >>         Can you add
>>>     >>
>>>     >>                         <session>
>>>     >>                             <ejb-name>TradeJDBC</ejb-name>
>>>     >>                             <jndi-name>ejb/TradeJDBC</jndi-name>
>>>     >>                             <cache-size>100</cache-size>
>>>     >>                         </session>
>>>     >>
>>>     >>         to your plan and redeploy.  I added some support for
>>>     multiple
>>>     >>         SLSBs in a pool for 1.2 which we did not have 
>>> before.  This
>>>     >>         will hopefully make it better and not worse :)
>>>     >>
>>>     >>         On Apr 6, 2007, at 11:32 AM, Christopher Blythe wrote:
>>>     >>
>>>     >>>         Matt...
>>>     >>>
>>>     >>>         You mentioned that you deployed DayTrader 1.2... did

>>> you
>>>     >>>         happen to run it under load? JDBC/Direct mode looks

>>> good;
>>>     >>>         however, I am still seeing
>>>     ConcurrentModificationExceptions
>>>     >>>         while attempting to run more than 1 client in Session
>>>     Direct
>>>     >>>         mode (
>>>     https://issues.apache.org/jira/browse/GERONIMO-2708).
>>>     >>>         These exceptions are thrown throughout the duration

>>> of the
>>>     >>>         run. FYI - I deployed the same ear on Geronimo 1.1.1

>>> and
>>>     >>>         didn't have a problem scaling up the users for Session
>>>     >>>         Direct mode.
>>>     >>>
>>>     >>>         java.util.ConcurrentModificationException
>>>     >>>             at
>>>     java.util.HashMap$HashIterator.remove(HashMap.java:861)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.connector.outbound.connectiontracking.ConnectionTrackingCoordinator.exit

>>>
>>>     >>>         ( ConnectionTrackingCoordinator.java :127)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.connector.outbound.connectiontracking.ConnectionTrackingCoordinator$$FastClassByCGLIB$$5d33aabf.invoke(<generated>)

>>>
>>>
>>>     >>>             at net.sf.cglib.reflect.FastMethod.invoke
>>>     >>>         (FastMethod.java :53)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke
>>>     (FastMethodInvoker.java:38)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122)

>>>
>>>     >>>             at
>>>     >>>         org.apache.geronimo.gbean.runtime.GBeanInstance.invoke
>>>     >>>         (GBeanInstance.java:820)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35)

>>>
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept
>>>     >>>         ( ProxyMethodInterceptor.java:96)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.connector.outbound.connectiontracking.ConnectionTracker$$EnhancerByCGLIB$$b6b1324a.exit(<generated>)

>>>
>>>     >>>             at
>>>     >>>         
>>> org.apache.openejb.NoConnectionEnlistingInterceptor.invoke
>>>     >>>         (NoConnectionEnlistingInterceptor.java:70)
>>>     >>>             at
>>>     >>>         org.apache.openejb.SystemExceptionInterceptor.invoke
>>>     (SystemExceptionInterceptor.java:35)
>>>     >>>             at
>>>     >>>            
>>> org.apache.openejb.security.DefaultSubjectInterceptor.invoke(DefaultSubjectInterceptor.java

>>>
>>>     >>>         :49)
>>>     >>>             at
>>>     >>>            
>>> org.apache.openejb.slsb.DefaultStatelessEjbContainer.invoke(DefaultStatelessEjbContainer.java:178)

>>>
>>>     >>>             at
>>>     >>>            
>>> org.apache.openejb.slsb.DefaultStatelessEjbContainer$$FastClassByCGLIB$$7ad7a562.invoke

>>>
>>>     (<generated>)
>>>     >>>
>>>     >>>             at
>>>     >>>         
>>> net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke
>>>     (FastMethodInvoker.java:38)
>>>     >>>             at
>>>     >>>         org.apache.geronimo.gbean.runtime.GBeanOperation.invoke
>>>     >>>         (GBeanOperation.java:122)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:820)

>>>
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java
>>>     :57)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke
>>>     >>>         (RawOperationInvoker.java:35)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96)

>>>
>>>     >>>             at
>>>     >>>            
>>> org.apache.openejb.StatelessEjbContainer$$EnhancerByCGLIB$$5c554f35.invoke 
>>>
>>>
>>>     >>>         (<generated>)
>>>     >>>             at
>>>     >>>            
>>> org.apache.openejb.AbstractEjbDeployment.invoke(AbstractEjbDeployment.java:195)

>>>
>>>     >>>             at
>>>     >>>            
>>> org.apache.openejb.proxy.EJBMethodInterceptor.intercept(EJBMethodInterceptor.java:145)

>>>
>>>     >>>             at
>>>     >>>            
>>> org.apache.openejb.proxy.SessionEJBObject$$EnhancerByCGLIB$$f5a9c1b2.login 
>>>
>>>     >>>         (<generated>)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.samples.daytrader.TradeAction.login(TradeAction.java:449)

>>>
>>>     >>>             at org.apache.geronimo.samples.daytrader.
>>>     >>>         web.TradeServletAction.doLogin
>>>     >>>            
>>> <http://web.TradeServletAction.doLogin>(TradeServletAction.java:364)
>>>     >>>             at org.apache.geronimo.samples.daytrader .
>>>     >>>         web.TradeAppServlet.performTask
>>>     >>>            
>>> <http://web.TradeAppServlet.performTask>(TradeAppServlet.java:126)
>>>     >>>             at org.apache.geronimo.samples.daytrader.
>>>     >>>         web.TradeAppServlet.doPost
>>>     >>>            
>>> <http://web.TradeAppServlet.doPost>(TradeAppServlet.java :91)
>>>     >>>             at javax.servlet.http.HttpServlet.service
>>>     >>>         (HttpServlet.java:617)
>>>     >>>             at
>>>     >>>            
>>> javax.servlet.http.HttpServlet.service(HttpServlet.java :690)
>>>     >>>             at
>>>     >>>            
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>>>     >>>         (ApplicationFilterChain.java:252)
>>>     >>>             at
>>>     >>>         
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter
>>>     >>>         (ApplicationFilterChain.java:173)
>>>     >>>             at org.apache.geronimo.samples.daytrader.
>>>     >>>         web.OrdersAlertFilter.doFilter
>>>     >>>            
>>> <http://web.OrdersAlertFilter.doFilter>(OrdersAlertFilter.java:91)
>>>     >>>             at
>>>     >>>            
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>>>     (ApplicationFilterChain.java
>>>     >>>         :202)
>>>     >>>             at
>>>     >>>         
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter
>>>     >>>         (ApplicationFilterChain.java :173)
>>>     >>>             at
>>>     >>>            
>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)

>>>
>>>     >>>             at
>>>     org.apache.catalina.core.StandardContextValve.invoke
>>>     >>>         (StandardContextValve.java:178)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.tomcat.valve.DefaultSubjectValve.invoke(DefaultSubjectValve.java:56)

>>>
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.tomcat.GeronimoStandardContext$SystemMethodValve.invoke(GeronimoStandardContext.java

>>>
>>>     >>>         :328)
>>>     >>>             at
>>>     >>>            
>>> org.apache.geronimo.tomcat.valve.GeronimoBeforeAfterValve.invoke
>>>     (GeronimoBeforeAfterValve.java:47)
>>>     >>>             at
>>>     >>>            
>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)

>>>
>>>     >>>             at 
>>> org.apache.catalina.valves.ErrorReportValve.invoke
>>>     >>>         (ErrorReportValve.java:105)
>>>     >>>             at
>>>     >>>            
>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)

>>>
>>>     >>>             at
>>>     >>>            
>>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541) 
>>>
>>>     >>>             at 
>>> org.apache.catalina.connector.CoyoteAdapter.service
>>>     >>>         (CoyoteAdapter.java :148)
>>>     >>>             at
>>>     >>>            
>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) 
>>>
>>>     >>>             at
>>>     >>>            
>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection

>>>
>>>     (Http11BaseProtocol.java
>>>     >>>         :667)
>>>     >>>             at
>>>     >>>            
>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)

>>>
>>>     >>>             at
>>>     >>>            
>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)

>>>
>>>     >>>             at
>>>     >>>            
>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run
>>>     >>>         (ThreadPool.java:684)
>>>     >>>             at java.lang.Thread.run(Thread.java:797)
>>>     >>>
>>>     >>>         On 4/5/07, *Jason Dillon* < jason@planet57.com
>>>     <mailto:jason@planet57.com>
>>>     >>>         <mailto:jason@planet57.com
>>>     <mailto:jason@planet57.com>>> wrote:
>>>     >>>
>>>     >>>             Aight, no worries.  I still don't fully understand
>>>     all
>>>     >>>             that plugin stuff... yet ;-)
>>>     >>>
>>>     >>>             --jason
>>>     >>>
>>>     >>>
>>>     >>>             On Apr 5, 2007, at 3:38 PM, Paul McMahan wrote:
>>>     >>>
>>>     >>>>             The change I have cued up replaces "
>>>     1.2-SNAPSHOT" with
>>>     >>>>             " 1.2" for all the catalog entries.  So it would
>>>     break
>>>     >>>>             anyone using the Geronimo plugin repo from a
>>>     >>>>             1.2-SNAPSHOT server (maybe not a huge deal).
 
>>> Also,
>>>     >>>>             I've tested the catalog updates by looping http
>>>     >>>>             requests to repo1.maven.org/maven2
>>>     <http://repo1.maven.org/maven2>
>>>     >>>>             <http://repo1.maven.org/maven2> back to
my local
>>>     maven
>>>     >>>>             repo.  So I've made some assumptions about the

>>> repo
>>>     >>>>             layout that should probably be verified.
>>>     >>>>
>>>     >>>>             Best wishes,
>>>     >>>>             Paul
>>>     >>>>
>>>     >>>>             On Apr 5, 2007, at 6:22 PM, Jason Dillon wrote:
>>>     >>>>
>>>     >>>>>             Will it hurt anything to commit it now?
 Or 
>>> will it
>>>     >>>>>             break things?
>>>     >>>>>
>>>     >>>>>             --jason
>>>     >>>>>
>>>     >>>>>
>>>     >>>>>             On Apr 5, 2007, at 3:14 PM, Paul McMahan
wrote:
>>>     >>>>>
>>>     >>>>>>
>>>     >>>>>>             On Apr 5, 2007, at 2:11 PM, Joe Bohn
wrote:
>>>     >>>>>>
>>>     >>>>>>>             I couldn't do much with the framework
assembly
>>>     as it
>>>     >>>>>>>             requires a plugin repository with
1.2 
>>> plugins and
>>>     >>>>>>>             AFAIK there is no such plugin repository

>>> available
>>>     >>>>>>>             yet.  Will you be making the plugins

>>> available for
>>>     >>>>>>>             1.2 as you make the release available?
 If
>>>     not, then
>>>     >>>>>>>             perhaps we shouldn't include the
framework
>>>     assembly
>>>     >>>>>>>             in the distribution.
>>>     >>>>>>
>>>     >>>>>>             I updated the plugin catalog stuff in
>>>     >>>>>>             site/trunk/docs/plugins/geronimo- 1.2
locally
>>>     and ran
>>>     >>>>>>             some quick tests of plugin download
& install 
>>> from
>>>     >>>>>>             maven repo.  I'm ready to commit if/when
the 1.2
>>>     >>>>>>             artifacts are published to central.
>>>     >>>>>>
>>>     >>>>>>             Best wishes,
>>>     >>>>>>             Paul
>>>     >>>>>>
>>>     >>>>>
>>>     >>>>
>>>     >>>
>>>     >>>
>>>     >>>
>>>     >>>
>>>     >>>         --
>>>     >>>         "I say never be complete, I say stop being perfect,

>>> I say
>>>     >>>         let... lets evolve, let the chips fall where they 
>>> may." -
>>>     >>>         Tyler Durden
>>>     >>
>>>     >>
>>>     >>
>>>     >>
>>>     >>     --
>>>     >>     "I say never be complete, I say stop being perfect, I say
>>>     let...
>>>     >>     lets evolve, let the chips fall where they may." - Tyler 
>>> Durden
>>>     >>     <geronimo.log>
>>>     >
>>>     >
>>>     >
>>>     >
>>>     > --
>>>     > "I say never be complete, I say stop being perfect, I say let...
>>>     lets
>>>     > evolve, let the chips fall where they may." - Tyler Durden
>>>
>>>
>>>
>>>
>>> --"I say never be complete, I say stop being perfect, I say let... 
>>> lets evolve, let the chips fall where they may." - Tyler Durden
>> Index: 
>> /usr/src/geronimo-1.2/modules/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTrackingCoordinator.java

>>
>> ===================================================================
>> --- 
>> /usr/src/geronimo-1.2/modules/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTrackingCoordinator.java
   
>> (revision 526213)
>> +++ 
>> /usr/src/geronimo-1.2/modules/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTrackingCoordinator.java
   
>> (working copy)
>> @@ -124,7 +124,23 @@
>>
>>                  // if no connection remain clear context... we could 
>> support automatic commit, rollback or exception here
>>                  if (connections.isEmpty()) {
>> -                    i.remove();
>> +                    boolean retry = false;
>> +                    int numberOfRetries = 0;
>> +                    do {
>> +                        try {
>> +                            i.remove();
>> +                        } catch 
>> (java.util.ConcurrentModificationException ex) {
>> +                            if (numberOfRetries < 5) {
>> +                                retry = true;
>> +                            } else {
>> +                                retry = false;
>> +                            }
>> +                            numberOfRetries += 1;
>> +                        }
>> +                    } while (retry);
>> +                    if (numberOfRetries >= 5) {
>> +                        throw new 
>> ResourceException("ConcurrentModificationException - Unable to remove 
>> resource");
>> +                    }
>>                  }
>>              }
>>          } finally {
>
>
>
>

Mime
View raw message