geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jay D. McHugh" <...@joyfulnoisewebdesign.com>
Subject Re: [discuss] Release Geronimo 1.2
Date Sat, 07 Apr 2007 02:55:06 GMT
Wait (crap)

This change builds, but fails the openejb tests

(forgot to run those before sending out the patch for confirmation)

Jay D. McHugh wrote:
> Hello all,
>
> New attempt that doesn't just hide the problem.
>
> I managed to find reference on how to make a linked list behave as 
> synchronized (above and beyond simply trying to access them from 
> within synchronized code blocks).
>
> Attached is the patch - It is actually for OpenEJB...I'm going to wait 
> until someone has a chance to confirm that it actually resolves the 
> Geronimo/Daytrader issue then make the JIRA over on OpenEJB.
>
> Jay
>
> David Jencks wrote:
>> I don't think this is acceptable.  There should be only one thread 
>> working with the context at a time.  Either this exception is caused 
>> by modifying the collection in the same thread in which case we can 
>> fix it easily or it is caused by more than one thread having access 
>> to a context at once.  Since by my reading of the code this is a 
>> context attached to a stateless session bean instance, that would 
>> mean that more than one thread is using a stateless session bean 
>> instance at once, which is definitely cause to -1 the release.
>>
>> I hope there's another possibility I haven't thought of..... but 
>> hiding the problem is not acceptable unless we really understand what 
>> is going on and are really convinced it's harmless.
>>
>> thanks
>> david jencks
>>
>> On Apr 6, 2007, at 1:40 PM, Jay D. McHugh wrote:
>>
>>> Chris (do you go by Chris or Christopher?),
>>>
>>> Here is a patch that I just wrote that allows the exit routine of 
>>> ConnectionTrackingCoordinator to finish cleanly after a number (5) 
>>> of attempts at removing the resource.
>>>
>>> If it fails after five tries, then the routine exits and throws a 
>>> ResourceException (that will hopefully be caught further up the stack).
>>>
>>> Would you like to try it to see if it solves your concurrency problem?
>>>
>>> Jay
>>>
>>> Christopher Blythe wrote:
>>>> Doubtful... everything tested fine under light browser based 
>>>> testing. As the exception suggests, this is a concurrency problem 
>>>> that you would only hit under load.
>>>>
>>>> On 4/6/07, * Jay D. McHugh* <jay@joyfulnoisewebdesign.com 
>>>> <mailto:jay@joyfulnoisewebdesign.com>> wrote:
>>>>
>>>>     If Matt had no problem deploying and testing DT, could it be a 
>>>> Java
>>>>     version or classpath issue?
>>>>
>>>>     That could explain the difference in the exception during 
>>>> deployment
>>>>     (and the problems during deployment could possibly explain the run
>>>>     time
>>>>     problems).
>>>>
>>>>     Jay
>>>>
>>>>     Christopher Blythe wrote:
>>>>     > I use a commercial load driving tool... FYI, I'm fairly 
>>>> certain that
>>>>     > G-2.0 has the same issue.
>>>>     >
>>>>     > On 4/6/07, *David Jencks* < david_jencks@yahoo.com
>>>>     <mailto:david_jencks@yahoo.com>
>>>>     > <mailto:david_jencks@yahoo.com <mailto:david_jencks@yahoo.com>>>
>>>>     wrote:
>>>>     >
>>>>     >     I think we need to figure out why the
>>>>     >     concurrentModificationException is happening before we
>>>>     release.  I
>>>>     >     think that one possible reason is that we are multithreading
>>>>     >     stateless session bean instances.  I hope this isn't the
>>>>     cause....
>>>>     >     but IMO we need to find out.
>>>>     >
>>>>     >     Chris, how do you run the several clients?  manually or with
>>>>     a tool?
>>>>     >
>>>>     >     thanks
>>>>     >     david jencks
>>>>     >
>>>>     >
>>>>     >     On Apr 6, 2007, at 11:09 AM, Christopher Blythe wrote:
>>>>     >
>>>>     >>     Gave it a shot... no luck. As soon as I started 2 
>>>> clients, the
>>>>     >>     same exceptions started to pile up. I have attached the
>>>>     >>     geronimo.log. Also, noticed the following exception during
>>>>     startup.
>>>>     >>
>>>>     >>     14:05:00,640 ERROR [TransportConnector] Could not accept
>>>>     >>     connection from /127.0.0.1:28428: java.io.IOException: Wire
>>>>     >>     format negociation timeout: peer did not send his wire 
>>>> format.
>>>>     >>     java.io.IOException: Wire format negociation timeout: 
>>>> peer did
>>>>     >>     not send his wire format.
>>>>     >>         at
>>>>     org.apache.activemq.transport.WireFormatNegotiator.oneway
>>>>     >>     (WireFormatNegotiator.java :88)
>>>>     >>         at
>>>>     >>        
>>>> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:47)

>>>>
>>>>     >>         at 
>>>> org.apache.activemq.broker.TransportConnection.dispatch
>>>>     >>     (TransportConnection.java :1138)
>>>>     >>         at
>>>>     >>        
>>>> org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:805)

>>>>
>>>>     >>         at
>>>>     >>     org.apache.activemq.broker.TransportConnection.start
>>>>     (TransportConnection.java
>>>>     >>     :885)
>>>>     >>         at 
>>>> org.apache.activemq.broker.TransportConnector$1.onAccept
>>>>     >>     (TransportConnector.java:148)
>>>>     >>         at
>>>>     >>     org.apache.activemq.transport.tcp.TcpTransportServer.run
>>>>     (TcpTransportServer.java:167)
>>>>     >>         at java.lang.Thread.run (Thread.java:797)
>>>>     >>
>>>>     >>
>>>>     >>
>>>>     >>     On 4/6/07, *Matt Hogstrom* < matt@hogstrom.org
>>>>     <mailto:matt@hogstrom.org>
>>>>     >>     <mailto:matt@hogstrom.org <mailto:matt@hogstrom.org>>>

>>>> wrote:
>>>>     >>
>>>>     >>         Only a very light load from a few browsers.  One thing
>>>>     to try
>>>>     >>         is to increase the number of SLSBs in the pool.
>>>>     >>
>>>>     >>         Can you add
>>>>     >>
>>>>     >>                         <session>
>>>>     >>                             <ejb-name>TradeJDBC</ejb-name>
>>>>     >>                             
>>>> <jndi-name>ejb/TradeJDBC</jndi-name>
>>>>     >>                             <cache-size>100</cache-size>
>>>>     >>                         </session>
>>>>     >>
>>>>     >>         to your plan and redeploy.  I added some support for
>>>>     multiple
>>>>     >>         SLSBs in a pool for 1.2 which we did not have 
>>>> before.  This
>>>>     >>         will hopefully make it better and not worse :)
>>>>     >>
>>>>     >>         On Apr 6, 2007, at 11:32 AM, Christopher Blythe wrote:
>>>>     >>
>>>>     >>>         Matt...
>>>>     >>>
>>>>     >>>         You mentioned that you deployed DayTrader 1.2...

>>>> did you
>>>>     >>>         happen to run it under load? JDBC/Direct mode looks

>>>> good;
>>>>     >>>         however, I am still seeing
>>>>     ConcurrentModificationExceptions
>>>>     >>>         while attempting to run more than 1 client in Session
>>>>     Direct
>>>>     >>>         mode (
>>>>     https://issues.apache.org/jira/browse/GERONIMO-2708).
>>>>     >>>         These exceptions are thrown throughout the duration

>>>> of the
>>>>     >>>         run. FYI - I deployed the same ear on Geronimo 
>>>> 1.1.1 and
>>>>     >>>         didn't have a problem scaling up the users for Session
>>>>     >>>         Direct mode.
>>>>     >>>
>>>>     >>>         java.util.ConcurrentModificationException
>>>>     >>>             at
>>>>     java.util.HashMap$HashIterator.remove(HashMap.java:861)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.connector.outbound.connectiontracking.ConnectionTrackingCoordinator.exit

>>>>
>>>>     >>>         ( ConnectionTrackingCoordinator.java :127)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.connector.outbound.connectiontracking.ConnectionTrackingCoordinator$$FastClassByCGLIB$$5d33aabf.invoke(<generated>)

>>>>
>>>>
>>>>     >>>             at net.sf.cglib.reflect.FastMethod.invoke
>>>>     >>>         (FastMethod.java :53)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke
>>>>     (FastMethodInvoker.java:38)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.gbean.runtime.GBeanOperation.invoke(GBeanOperation.java:122)

>>>>
>>>>     >>>             at
>>>>     >>>         org.apache.geronimo.gbean.runtime.GBeanInstance.invoke
>>>>     >>>         (GBeanInstance.java:820)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java:57) 
>>>>
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke(RawOperationInvoker.java:35)

>>>>
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept
>>>>     >>>         ( ProxyMethodInterceptor.java:96)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.connector.outbound.connectiontracking.ConnectionTracker$$EnhancerByCGLIB$$b6b1324a.exit(<generated>)

>>>>
>>>>     >>>             at
>>>>     >>>         
>>>> org.apache.openejb.NoConnectionEnlistingInterceptor.invoke
>>>>     >>>         (NoConnectionEnlistingInterceptor.java:70)
>>>>     >>>             at
>>>>     >>>         org.apache.openejb.SystemExceptionInterceptor.invoke
>>>>     (SystemExceptionInterceptor.java:35)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.openejb.security.DefaultSubjectInterceptor.invoke(DefaultSubjectInterceptor.java

>>>>
>>>>     >>>         :49)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.openejb.slsb.DefaultStatelessEjbContainer.invoke(DefaultStatelessEjbContainer.java:178)

>>>>
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.openejb.slsb.DefaultStatelessEjbContainer$$FastClassByCGLIB$$7ad7a562.invoke

>>>>
>>>>     (<generated>)
>>>>     >>>
>>>>     >>>             at
>>>>     >>>         
>>>> net.sf.cglib.reflect.FastMethod.invoke(FastMethod.java:53)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.gbean.runtime.FastMethodInvoker.invoke
>>>>     (FastMethodInvoker.java:38)
>>>>     >>>             at
>>>>     >>>         
>>>> org.apache.geronimo.gbean.runtime.GBeanOperation.invoke
>>>>     >>>         (GBeanOperation.java:122)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.gbean.runtime.GBeanInstance.invoke(GBeanInstance.java:820)

>>>>
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.gbean.runtime.RawInvoker.invoke(RawInvoker.java
>>>>     :57)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.kernel.basic.RawOperationInvoker.invoke
>>>>     >>>         (RawOperationInvoker.java:35)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.kernel.basic.ProxyMethodInterceptor.intercept(ProxyMethodInterceptor.java:96)

>>>>
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.openejb.StatelessEjbContainer$$EnhancerByCGLIB$$5c554f35.invoke

>>>>
>>>>
>>>>     >>>         (<generated>)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.openejb.AbstractEjbDeployment.invoke(AbstractEjbDeployment.java:195)

>>>>
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.openejb.proxy.EJBMethodInterceptor.intercept(EJBMethodInterceptor.java:145)

>>>>
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.openejb.proxy.SessionEJBObject$$EnhancerByCGLIB$$f5a9c1b2.login

>>>>
>>>>     >>>         (<generated>)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.samples.daytrader.TradeAction.login(TradeAction.java:449)

>>>>
>>>>     >>>             at org.apache.geronimo.samples.daytrader.
>>>>     >>>         web.TradeServletAction.doLogin
>>>>     >>>            
>>>> <http://web.TradeServletAction.doLogin>(TradeServletAction.java:364)
>>>>     >>>             at org.apache.geronimo.samples.daytrader .
>>>>     >>>         web.TradeAppServlet.performTask
>>>>     >>>            
>>>> <http://web.TradeAppServlet.performTask>(TradeAppServlet.java:126)
>>>>     >>>             at org.apache.geronimo.samples.daytrader.
>>>>     >>>         web.TradeAppServlet.doPost
>>>>     >>>            
>>>> <http://web.TradeAppServlet.doPost>(TradeAppServlet.java :91)
>>>>     >>>             at javax.servlet.http.HttpServlet.service
>>>>     >>>         (HttpServlet.java:617)
>>>>     >>>             at
>>>>     >>>            
>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java :690)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>>>>     >>>         (ApplicationFilterChain.java:252)
>>>>     >>>             at
>>>>     >>>         
>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter
>>>>     >>>         (ApplicationFilterChain.java:173)
>>>>     >>>             at org.apache.geronimo.samples.daytrader.
>>>>     >>>         web.OrdersAlertFilter.doFilter
>>>>     >>>            
>>>> <http://web.OrdersAlertFilter.doFilter>(OrdersAlertFilter.java:91)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>>>>     (ApplicationFilterChain.java
>>>>     >>>         :202)
>>>>     >>>             at
>>>>     >>>         
>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter
>>>>     >>>         (ApplicationFilterChain.java :173)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)

>>>>
>>>>     >>>             at
>>>>     org.apache.catalina.core.StandardContextValve.invoke
>>>>     >>>         (StandardContextValve.java:178)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.tomcat.valve.DefaultSubjectValve.invoke(DefaultSubjectValve.java:56)

>>>>
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.tomcat.GeronimoStandardContext$SystemMethodValve.invoke(GeronimoStandardContext.java

>>>>
>>>>     >>>         :328)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.geronimo.tomcat.valve.GeronimoBeforeAfterValve.invoke
>>>>     (GeronimoBeforeAfterValve.java:47)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)

>>>>
>>>>     >>>             at 
>>>> org.apache.catalina.valves.ErrorReportValve.invoke
>>>>     >>>         (ErrorReportValve.java:105)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)

>>>>
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)

>>>>
>>>>     >>>             at 
>>>> org.apache.catalina.connector.CoyoteAdapter.service
>>>>     >>>         (CoyoteAdapter.java :148)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)

>>>>
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection

>>>>
>>>>     (Http11BaseProtocol.java
>>>>     >>>         :667)
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)

>>>>
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)

>>>>
>>>>     >>>             at
>>>>     >>>            
>>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run
>>>>     >>>         (ThreadPool.java:684)
>>>>     >>>             at java.lang.Thread.run(Thread.java:797)
>>>>     >>>
>>>>     >>>         On 4/5/07, *Jason Dillon* < jason@planet57.com
>>>>     <mailto:jason@planet57.com>
>>>>     >>>         <mailto:jason@planet57.com
>>>>     <mailto:jason@planet57.com>>> wrote:
>>>>     >>>
>>>>     >>>             Aight, no worries.  I still don't fully understand
>>>>     all
>>>>     >>>             that plugin stuff... yet ;-)
>>>>     >>>
>>>>     >>>             --jason
>>>>     >>>
>>>>     >>>
>>>>     >>>             On Apr 5, 2007, at 3:38 PM, Paul McMahan wrote:
>>>>     >>>
>>>>     >>>>             The change I have cued up replaces "
>>>>     1.2-SNAPSHOT" with
>>>>     >>>>             " 1.2" for all the catalog entries.  So
it would
>>>>     break
>>>>     >>>>             anyone using the Geronimo plugin repo from
a
>>>>     >>>>             1.2-SNAPSHOT server (maybe not a huge deal).
 
>>>> Also,
>>>>     >>>>             I've tested the catalog updates by looping
http
>>>>     >>>>             requests to repo1.maven.org/maven2
>>>>     <http://repo1.maven.org/maven2>
>>>>     >>>>             <http://repo1.maven.org/maven2> back
to my local
>>>>     maven
>>>>     >>>>             repo.  So I've made some assumptions about
the 
>>>> repo
>>>>     >>>>             layout that should probably be verified.
>>>>     >>>>
>>>>     >>>>             Best wishes,
>>>>     >>>>             Paul
>>>>     >>>>
>>>>     >>>>             On Apr 5, 2007, at 6:22 PM, Jason Dillon
wrote:
>>>>     >>>>
>>>>     >>>>>             Will it hurt anything to commit it now?
 Or 
>>>> will it
>>>>     >>>>>             break things?
>>>>     >>>>>
>>>>     >>>>>             --jason
>>>>     >>>>>
>>>>     >>>>>
>>>>     >>>>>             On Apr 5, 2007, at 3:14 PM, Paul McMahan
wrote:
>>>>     >>>>>
>>>>     >>>>>>
>>>>     >>>>>>             On Apr 5, 2007, at 2:11 PM, Joe
Bohn wrote:
>>>>     >>>>>>
>>>>     >>>>>>>             I couldn't do much with the
framework assembly
>>>>     as it
>>>>     >>>>>>>             requires a plugin repository
with 1.2 
>>>> plugins and
>>>>     >>>>>>>             AFAIK there is no such plugin
repository 
>>>> available
>>>>     >>>>>>>             yet.  Will you be making the
plugins 
>>>> available for
>>>>     >>>>>>>             1.2 as you make the release
available?  If
>>>>     not, then
>>>>     >>>>>>>             perhaps we shouldn't include
the framework
>>>>     assembly
>>>>     >>>>>>>             in the distribution.
>>>>     >>>>>>
>>>>     >>>>>>             I updated the plugin catalog stuff
in
>>>>     >>>>>>             site/trunk/docs/plugins/geronimo-
1.2 locally
>>>>     and ran
>>>>     >>>>>>             some quick tests of plugin download
& 
>>>> install from
>>>>     >>>>>>             maven repo.  I'm ready to commit
if/when the 
>>>> 1.2
>>>>     >>>>>>             artifacts are published to central.
>>>>     >>>>>>
>>>>     >>>>>>             Best wishes,
>>>>     >>>>>>             Paul
>>>>     >>>>>>
>>>>     >>>>>
>>>>     >>>>
>>>>     >>>
>>>>     >>>
>>>>     >>>
>>>>     >>>
>>>>     >>>         --
>>>>     >>>         "I say never be complete, I say stop being perfect,

>>>> I say
>>>>     >>>         let... lets evolve, let the chips fall where they

>>>> may." -
>>>>     >>>         Tyler Durden
>>>>     >>
>>>>     >>
>>>>     >>
>>>>     >>
>>>>     >>     --
>>>>     >>     "I say never be complete, I say stop being perfect, I say
>>>>     let...
>>>>     >>     lets evolve, let the chips fall where they may." - Tyler

>>>> Durden
>>>>     >>     <geronimo.log>
>>>>     >
>>>>     >
>>>>     >
>>>>     >
>>>>     > --
>>>>     > "I say never be complete, I say stop being perfect, I say let...
>>>>     lets
>>>>     > evolve, let the chips fall where they may." - Tyler Durden
>>>>
>>>>
>>>>
>>>>
>>>> --"I say never be complete, I say stop being perfect, I say let... 
>>>> lets evolve, let the chips fall where they may." - Tyler Durden
>>> Index: 
>>> /usr/src/geronimo-1.2/modules/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTrackingCoordinator.java

>>>
>>> ===================================================================
>>> --- 
>>> /usr/src/geronimo-1.2/modules/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTrackingCoordinator.java
   
>>> (revision 526213)
>>> +++ 
>>> /usr/src/geronimo-1.2/modules/geronimo-connector/src/main/java/org/apache/geronimo/connector/outbound/connectiontracking/ConnectionTrackingCoordinator.java
   
>>> (working copy)
>>> @@ -124,7 +124,23 @@
>>>
>>>                  // if no connection remain clear context... we 
>>> could support automatic commit, rollback or exception here
>>>                  if (connections.isEmpty()) {
>>> -                    i.remove();
>>> +                    boolean retry = false;
>>> +                    int numberOfRetries = 0;
>>> +                    do {
>>> +                        try {
>>> +                            i.remove();
>>> +                        } catch 
>>> (java.util.ConcurrentModificationException ex) {
>>> +                            if (numberOfRetries < 5) {
>>> +                                retry = true;
>>> +                            } else {
>>> +                                retry = false;
>>> +                            }
>>> +                            numberOfRetries += 1;
>>> +                        }
>>> +                    } while (retry);
>>> +                    if (numberOfRetries >= 5) {
>>> +                        throw new 
>>> ResourceException("ConcurrentModificationException - Unable to 
>>> remove resource");
>>> +                    }
>>>                  }
>>>              }
>>>          } finally {
>>
>>
>>
>>

Mime
View raw message