cxf-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Gregory <GGreg...@seagullsoftware.com>
Subject Re: ConcurrentModificationException on Client when processing SOAPHeaders....
Date Wed, 27 Jul 2011 19:57:58 GMT
Hi Dan,

Any thoughts on the timing for 2.4.2?

Gary

On Jul 27, 2011, at 15:34, "Daniel Kulp" <dkulp@apache.org> wrote:

> 
> Just looked at the code....
> 
> 
> The JaxWsProxyFactoryBean doesn't have the synchronized flag on the create 
> method like the one on the superclass does.   :-(
> 
> You can likely workaround this by creating your own subclass of 
> JaxWsProxyFactoryBean that just has a single method of:
> 
> 
> public synchronized Object create() {
>    return super.create();
> }
> 
> and using that instead.    I'll get this fixes for 2.4.2.  
> 
> Dan
> 
> 
> 
> 
> On Tuesday, July 26, 2011 10:17:10 PM Algirdas Veitas wrote:
>> OK, we migrated to 2.3.5 and are still using a "client per SOAP call"
>> strategy. We are still seeing intermittent issues.  We have tried to
>> configure our client as a spring bean with scope prototype AND call the
>> create() method in our code directly.  In both cases, we are getting
>> intermittent NPE's in JaxWsProxyFactoryBean, but on different lines.
>> Unfortunately, upgrading to 2.4.1 didn't resolve the issue either :(
>> 
>> We have a simple test case that has 2 threads....and here is the stack trace
>> when our client in configured as a spring bean (prototype)
>> 
>> org.springframework.beans.factory.BeanCreationException: Error creating bean
>> with name 'XService' defined in class path resource
>> [spring/myCxfClient.xml]: Instantiation of bean failed; nested exception is
>> org.springframework.beans.factory.BeanDefinitionStoreException: Factory
>> method [public java.lang.Object
>> org.apache.cxf.jaxws.JaxWsProxyFactoryBean.create()] threw exception; nested
>> exception is java.lang.NullPointerException
>> Index:0, about to call ws
>>    at
>> org.springframework.beans.factory.support.ConstructorResolver.instantiateUsi
>> ngFactoryMethod(ConstructorResolver.java:581) at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory
>> .instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:983)
>> at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory
>> .createBeanInstance(AbstractAutowireCapableBeanFactory.java:879) at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory
>> .doCreateBean(AbstractAutowireCapableBeanFactory.java:485) at
>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory
>> .createBean(AbstractAutowireCapableBeanFactory.java:456) at
>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(Abst
>> ractBeanFactory.java:310) at
>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(Abstra
>> ctBeanFactory.java:190) at
>> com.xHelperImpl.getXService(XServiceHelperImpl.java:203)
>>    at
>> com.helper.XHelperImplTest$ITUserReadOnlyClient.run(ITUserServiceHelperImplT
>> est.java:98) at java.lang.Thread.run(Thread.java:662)
>> Caused by: org.springframework.beans.factory.BeanDefinitionStoreException:
>> Factory method [public java.lang.Object
>> org.apache.cxf.jaxws.JaxWsProxyFactoryBean.create()] threw exception; nested
>> exception is java.lang.NullPointerException
>>    at
>> org.springframework.beans.factory.support.SimpleInstantiationStrategy.instan
>> tiate(SimpleInstantiationStrategy.java:157) at
>> org.springframework.beans.factory.support.ConstructorResolver.instantiateUsi
>> ngFactoryMethod(ConstructorResolver.java:570) ... 10 more
>> Caused by: java.lang.NullPointerException
>>    at
>> org.apache.cxf.jaxws.JaxWsProxyFactoryBean.needWrapperClassInterceptor(JaxWs
>> ProxyFactoryBean.java:152) at
>> org.apache.cxf.jaxws.JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java
>> :135) at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source) at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
>> .java:25) at java.lang.reflect.Method.invoke(Method.java:597)
>>    at
>> org.springframework.beans.factory.support.SimpleInstantiationStrategy.instan
>> tiate(SimpleInstantiationStrategy.java:145) ... 11 more
>> 
>> 
>> Here is our configuration....
>> 
>> <bean id="xServiceProxyFactory"
>> class="org.apache.cxf.jaxws.JaxWsProxyFactoryBean">
>>   <property name="serviceClass" value="com.XService"/>
>>   <property name="address" ref="xServiceUrl"/>
>> </bean>
>> 
>> <bean id="xService" scope="prototype" class="com.XService"
>> factory-bean="xServiceProxyFactory" factory-method="create"/>
>> 
>> 
>> When we call create() on the proxy factory in our code (and not use a
>> prototype scoped bean), we see a similiar error (but not as frequently as
>> the above) error:
>> 
>> java.lang.NullPointerException
>>    at
>> org.apache.cxf.jaxws.JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.java
>> :135) at com.xServiceHelperImpl.getXService(XServiceHelperImpl.java:210) at
>> com.helper.XHelperImplTest$ITUserReadOnlyClient.run(ITUserServiceHelperImplT
>> est.java:98)
>> 
>> 
>> Al
>> 
>> On Mon, Jul 25, 2011 at 4:49 PM, Daniel Kulp <dkulp@apache.org> wrote:
>>> On Monday, July 25, 2011 12:53:37 PM Algirdas Veitas wrote:
>>>> Will give that a shot, but it looks like this is the same issue as
>>>> https://issues.apache.org/jira/browse/CXF-3075, and according the
>>>> entry
>>> 
>>> was
>>> 
>>>> fixed in 2.3.1.  Were there more changes since then?
>>> 
>>> For the client creation, there is:
>>> https://issues.apache.org/jira/browse/CXF-3558
>>> 
>>> which  is in 2.3.5.
>>> 
>>> Ideally, you'd move right to 2.4.1.  It's definitely easier to debug
>>> things from my side there.
>>> 
>>> 
>>> Dan
>>> 
>>>> Al
>>>> 
>>>> On Mon, Jul 25, 2011 at 12:48 PM, Daniel Kulp <dkulp@apache.org> wrote:
>>>>> Can you move up to 2.3.5?   We did add some synchronized blocks
>>>>> around the ReflectionServiceFactoryBean to make the singleton
>>>>> cases work a lot better. That may fix this issue.
>>>>> 
>>>>> Dan
>>>>> 
>>>>> On Monday, July 25, 2011 12:35:11 PM Algirdas Veitas wrote:
>>>>>> More information....after doing some additional testing, it
>>>>>> looks
>>>>>> like
>>>>> 
>>>>> the
>>>>> 
>>>>>> exception is thrown in different areas of the CXF code, but
>>>>>> the
>>>>>> ConcurrentModificationException is thrown when trying to
>>>>>> manipulate
>>>>>> the
>>>>>> 
>>>>>>    private List<AbstractServiceConfiguration>
>>>>>>    serviceConfigurations =
>>>>>> 
>>>>>>        new
>>>>>>        ArrayList<AbstractServiceConfiguration>();
>>>>>> 
>>>>>> that belongs to ReflectionServiceFactoryBean....see stack
>>>>>> traces
>>>>>> below
>>>>> 
>>>>> for
>>>>> 
>>>>>> additional samples:
>>>>>> 
>>>>>> Caused by: java.util.ConcurrentModificationException
>>>>>> 
>>>>>>    at
>>>>>> 
>>>>>> java.util.AbstractList$Itr.checkForComodification(AbstractLi
>>>>>> st.java: 372)
>>>>>> 
>>>>>>    at
>>>>>>    java.util.AbstractList$Itr.next(AbstractList.java:3
>>>>>>    43)
>>>>>>    at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.createInputW
>>> 
>>>>> rapp
>>>>> 
>>>>>> edMessageParts(ReflectionServiceFactoryBean.java:1619) at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.createOperat
>>> 
>>>>> ion(
>>>>> 
>>>>>> ReflectionServiceFactoryBean.java:934) at
>>> 
>>> org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean.createOperation(Jax
>>> 
>>>>> WsSe
>>>>> 
>>>>>> rviceFactoryBean.java:616) at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.createInterf
>>> 
>>>>> ace(
>>>>> 
>>>>>> ReflectionServiceFactoryBean.java:907) at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.buildService
>>> 
>>>>> From
>>>>> 
>>>>>> Class(ReflectionServiceFactoryBean.java:433) at
>>> 
>>> org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean.buildServiceFromCla
>>> 
>>>>> ss(J
>>>>> 
>>>>>> axWsServiceFactoryBean.java:680) at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.initializeSe
>>> 
>>>>> rvic
>>>>> 
>>>>>> eModel(ReflectionServiceFactoryBean.java:505) at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.create(Refle
>>> 
>>>>> ctio
>>>>> 
>>>>>> nServiceFactoryBean.java:242) at
>>> 
>>> org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean.create(JaxWsService
>>> 
>>>>> Fact
>>>>> 
>>>>>> oryBean.java:202) at
>>> 
>>> org.apache.cxf.frontend.AbstractWSDLBasedEndpointFactory.createEndpoint(
>>> 
>>>>> Abst
>>>>> 
>>>>>> ractWSDLBasedEndpointFactory.java:101) at
>>> 
>>> org.apache.cxf.frontend.ClientFactoryBean.create(ClientFactoryBean.java:
>>>>> 90)
>>>>> 
>>>>>>    at
>>> 
>>> org.apache.cxf.frontend.ClientProxyFactoryBean.create(ClientProxyFactory
>>> 
>>>>> Bean
>>>>> 
>>>>>> .java:117
>>>>>> 
>>>>>> Caused by: java.util.ConcurrentModificationException
>>>>>> 
>>>>>>    at
>>>>>> 
>>>>>> java.util.AbstractList$Itr.checkForComodification(AbstractLi
>>>>>> st.java: 372)
>>>>>> 
>>>>>>    at
>>>>>>    java.util.AbstractList$Itr.next(AbstractList.java:3
>>>>>>    43)
>>>>>>    at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.addFault(Ref
>>> 
>>>>> lect
>>>>> 
>>>>>> ionServiceFactoryBean.java:2032) at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.initializeFa
>>> 
>>>>> ults
>>>>> 
>>>>>> (ReflectionServiceFactoryBean.java:2011) at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.createMessag
>>> 
>>>>> ePar
>>>>> 
>>>>>> ts(ReflectionServiceFactoryBean.java:1575) at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.createOperat
>>> 
>>>>> ion(
>>>>> 
>>>>>> ReflectionServiceFactoryBean.java:928) at
>>> 
>>> org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean.createOperation(Jax
>>> 
>>>>> WsSe
>>>>> 
>>>>>> rviceFactoryBean.java:616) at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.createInterf
>>> 
>>>>> ace(
>>>>> 
>>>>>> ReflectionServiceFactoryBean.java:907) at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.buildService
>>> 
>>>>> From
>>>>> 
>>>>>> Class(ReflectionServiceFactoryBean.java:433) at
>>> 
>>> org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean.buildServiceFromCla
>>> 
>>>>> ss(J
>>>>> 
>>>>>> axWsServiceFactoryBean.java:680) at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.initializeSe
>>> 
>>>>> rvic
>>>>> 
>>>>>> eModel(ReflectionServiceFactoryBean.java:505) at
>>> 
>>> org.apache.cxf.service.factory.ReflectionServiceFactoryBean.create(Refle
>>> 
>>>>> ctio
>>>>> 
>>>>>> nServiceFactoryBean.java:242) at
>>> 
>>> org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean.create(JaxWsService
>>> 
>>>>> Fact
>>>>> 
>>>>>> oryBean.java:202) at
>>> 
>>> org.apache.cxf.frontend.AbstractWSDLBasedEndpointFactory.createEndpoint(
>>> 
>>>>> Abst
>>>>> 
>>>>>> ractWSDLBasedEndpointFactory.java:101) at
>>> 
>>> org.apache.cxf.frontend.ClientFactoryBean.create(ClientFactoryBean.java:
>>>>> 90)
>>>>> 
>>>>>>    at
>>> 
>>> org.apache.cxf.frontend.ClientProxyFactoryBean.create(ClientProxyFactory
>>> 
>>>>> Bean
>>>>> 
>>>>>> .java:117) at
>>> 
>>> org.apache.cxf.jaxws.JaxWsProxyFactoryBean.create(JaxWsProxyFactoryBean.
>>> 
>>>>> java
>>>>> 
>>>>>> :124)
>>>>>> 
>>>>>> On Mon, Jul 25, 2011 at 11:30 AM, Algirdas Veitas
>>>>>> <apveitas@gmail.com
>>>>>> 
>>>>>> wrote:
>>>>>>> whoops...in the previous post...we mentioned 2.1.3 but
>>>>>>> we are
>>>>>>> using
>>>>>>> 2.3.1
>>>>>>> 
>>>>>>> On Mon, Jul 25, 2011 at 10:36 AM, Algirdas Veitas
>>>>> 
>>>>> <apveitas@gmail.com>wrote:
>>>>>>>> Hi Daniel,
>>>>>>>> 
>>>>>>>> Unfortunately, we are unable to reproduce this problem
>>>>>>>> in an
>>>>>>>> isolated environment.  But at this point, it does look
>>>>>>>> like
>>>>>>>> to be an issue with multiple threads using the same
>>>>>>>> instance
>>>>>>>> of the client.  Because we are
>>>>>>>> unable to reproduce, our current strategy is to create
>>>>>>>> a
>>>>>>>> separate
>>>>>>>> client per SOAP call.
>>>>>>>> 
>>>>>>>> Using the following documentation as a guideline
>>>>>>>> http://cxf.apache.org/docs/jax-ws-configuration.html,
>>>>>>>> we came
>>>>>>>> up with the following:
>>>>>>>> 
>>>>>>>> Here was the original configuration for the client:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> <jaxws:client id="xService" 
>>>>>>>> serviceClass="com.XService"
>>>>>>>> address="#xServiceUrl"
>>>>>>>> </jaxws:client>
>>>>>>>> 
>>>>>>>> Here is what we just tried:
>>>>>>>> 
>>>>>>>> <bean id="xServiceProxyFactory"
>>>>>>>> 
>>>>>>>>        class="org.apache.cxf.jaxws.
>>>>>>>> 
>>>>>>>> JaxWsProxyFactoryBean">
>>>>>>>> 
>>>>>>>>        <property name="serviceClass"
>>>>>>>>        value="com.XService"/>
>>>>>>>>        <property name="address"
>>>>>>>>        ref="xServiceUrl"/>
>>>>>>>> 
>>>>>>>> </bean>
>>>>>>>> 
>>>>>>>> <bean id="xService" scope="prototype"
>>>>>>>> class="com.XService"
>>>>>>>> 
>>>>>>>>              factory-bean="xServiceProx
>>>>>>>>              yFactory
>>>>>>>>              "
>>>>>>>> 
>>>>>>>> factory-method="create"/>
>>>>>>>> 
>>>>>>>> Notice that the xService bean is a prototype, so we
>>>>>>>> get a new
>>>>>>>> instance for each request made, to avoid the original
>>>>>>>> issue
>>>>>>>> we were seeing.
>>>>>>>> 
>>>>>>>> Now in our code we have a class that looks like this:
>>>>>>>> 
>>>>>>>> public class xServiceHelperImpl implements
>>>>>>>> BeanFactoryAware
>>>>>>>> {
>>>>>>>> 
>>>>>>>>    @Autowired
>>>>>>>>    @Qualifier(value="xServiceProxyFactory")
>>>>>>>>    private JaxWsProxyFactoryBean
>>>>>>>>    proxyFactoryBean;
>>>>>>>> 
>>>>>>>>    @Override
>>>>>>>>    public void setBeanFactory(BeanFactory
>>>>>>>>    beanFactory)
>>>>>>>>    throws
>>>>>>>> 
>>>>>>>> BeansException {
>>>>>>>> 
>>>>>>>>        this.beanFactory=beanFactory;
>>>>>>>> 
>>>>>>>>    }
>>>>>>>> 
>>>>>>>>    private xService getXService() {
>>>>>>>> 
>>>>>>>>        xService xService =
>>>>>>>> 
>>>>>>>> (xService)this.beanFactory.getBean("xService");
>>>>>>>> 
>>>>>>>>        return xService;
>>>>>>>> 
>>>>>>>>    }
>>>>>>>> 
>>>>>>>> }
>>>>>>>> 
>>>>>>>> And we have a JUnit test that spawns multiple threads
>>>>>>>> that
>>>>>>>> basically
>>>>>>>> calls "getXService()" and then invokes a web service
>>>>>>>> call.
>>>>>>>> 
>>>>>>>> When there is just 1 thread configured in the test,
>>>>>>>> everything
>>>>>>>> works
>>>>>>>> fine.
>>>>>>>> 
>>>>>>>> When there are 2 threads configured in the test, we
>>>>>>>> start
>>>>>>>> getting
>>>>>>>> errors, it is a ConcurrentModificationExcpeiton, but
>>>>>>>> in a
>>>>>>>> different
>>>>>>>> place then where we saw before :(....here is the stack
>>>>>>>> trace
>>>>>>>> 
>>>>>>>> org.springframework.beans.factory.BeanCreationExceptio
>>>>>>>> n: Error
>>>>>>>> creating
>>>>>>>> bean with name 'xService' defined in class path
>>>>>>>> resource
>>>>>>>> [spring/myCxfClient.xml]: Instantiation of bean
>>>>>>>> failed; nested
>>>>>>>> exception is
>>>>>>>> org.springframework.beans.factory.BeanDefinitionStoreE
>>>>>>>> xception
>>>>>>>> 
>>>>>>>> Factory method [public java.lang.Object
>>>>>>>> org.apache.cxf.jaxws.JaxWsProxyFactoryBean.create()]
>>>>>>>> threw
>>>>>>>> exception; nested exception is
>>>>>>>> java.util.ConcurrentModificationException
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.springframework.beans.factory.support.ConstructorR
>>>>>>>> esolver.
>>>>>>>> instanti
>>>>>>>> ateUsingFactoryMethod(ConstructorResolver.java:581)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.springframework.beans.factory.support.AbstractAuto
>>>>>>>> wireCapa
>>>>>>>> bleBeanF
>>>>>>>> actory.instantiateUsingFactoryMethod(AbstractAutowireC
>>>>>>>> apableB
>>>>>>>> eanFactor y.java:983)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.springframework.beans.factory.support.AbstractAuto
>>>>>>>> wireCapa
>>>>>>>> bleBeanF
>>>>>>>> actory.createBeanInstance(AbstractAutowireCapableBeanF
>>>>>>>> actory.
>>>>>>>> java:879)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.springframework.beans.factory.support.AbstractAuto
>>>>>>>> wireCapa
>>>>>>>> bleBeanF
>>>>>>>> actory.doCreateBean(AbstractAutowireCapableBeanFactory
>>>>>>>> .java:4
>>>>>>>> 85)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.springframework.beans.factory.support.AbstractAuto
>>>>>>>> wireCapa
>>>>>>>> bleBeanF
>>>>>>>> actory.createBean(AbstractAutowireCapableBeanFactory.j
>>>>>>>> ava:456
>>>>>>>> )
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.springframework.beans.factory.support.AbstractBean
>>>>>>>> Factory.
>>>>>>>> doGetBea n(AbstractBeanFactory.java:310)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.springframework.beans.factory.support.AbstractBean
>>>>>>>> Factory.
>>>>>>>> getBean( AbstractBeanFactory.java:190)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>>    com.XServiceHelperImpl.getXService(xServiceHel
>>>>>>>>    perImpl.
>>>>>>>>    java:21
>>>>>>>>    7)
>>>>>>>>    at
>>>>>>>>    com.XServiceHelperImpl.getX(xServiceHelperImpl
>>>>>>>>    .java:6
>>>>>>>>    3)
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> com..ITXServiceHelperImplTest$ITUserReadOnlyClient.run
>>>>>>>> (ITUserS
>>>>>>>> erviceHe lperImplTest.java:101)
>>>>>>>> 
>>>>>>>>    at java.lang.Thread.run(Thread.java:662)
>>>>>>>> 
>>>>>>>> Caused by:
>>>>>>>> org.springframework.beans.factory.BeanDefinitionStoreE
>>>>>>>> xception
>>>>>>>> 
>>>>>>>> Factory method [public java.lang.Object
>>>>>>>> org.apache.cxf.jaxws.JaxWsProxyFactoryBean.create()]
>>>>>>>> threw
>>>>>>>> exception; nested exception is
>>>>>>>> java.util.ConcurrentModificationException
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.springframework.beans.factory.support.SimpleInstan
>>>>>>>> tiationS
>>>>>>>> trategy.
>>>>>>>> instantiate(SimpleInstantiationStrategy.java:157)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.springframework.beans.factory.support.ConstructorR
>>>>>>>> esolver.
>>>>>>>> instanti
>>>>>>>> ateUsingFactoryMethod(ConstructorResolver.java:570)
>>>>>>>> 
>>>>>>>>    ... 10 more
>>>>>>>> 
>>>>>>>> Caused by: java.util.ConcurrentModificationException
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> java.util.AbstractList$Itr.checkForComodification(Abst
>>>>>>>> ractList
>>>>>>>> .java:37 2)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>>    java.util.AbstractList$Itr.next(AbstractList.j
>>>>>>>>    ava:343
>>>>>>>>    )
>>>>>>>> 
>>>>>>>>     at
>>>>>>>> 
>>>>>>>> org.apache.cxf.service.factory.ReflectionServiceFactor
>>>>>>>> yBean.cr
>>>>>>>> eateInpu
>>>>>>>> tWrappedMessageParts(ReflectionServiceFactoryBean.java
>>>>>>>> :1619)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.apache.cxf.service.factory.ReflectionServiceFactor
>>>>>>>> yBean.cr
>>>>>>>> eateOper ation(ReflectionServiceFactoryBean.java:934)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean.c
>>>>>>>> reateOpe
>>>>>>>> ration(J axWsServiceFactoryBean.java:616)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.apache.cxf.service.factory.ReflectionServiceFactor
>>>>>>>> yBean.cr
>>>>>>>> eateInte rface(ReflectionServiceFactoryBean.java:907)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.apache.cxf.service.factory.ReflectionServiceFactor
>>>>>>>> yBean.bu
>>>>>>>> ildServi
>>>>>>>> ceFromClass(ReflectionServiceFactoryBean.java:433)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean.b
>>>>>>>> uildServ
>>>>>>>> iceFromC lass(JaxWsServiceFactoryBean.java:680)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.apache.cxf.service.factory.ReflectionServiceFactor
>>>>>>>> yBean.in
>>>>>>>> itialize
>>>>>>>> ServiceModel(ReflectionServiceFactoryBean.java:505)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.apache.cxf.service.factory.ReflectionServiceFactor
>>>>>>>> yBean.cr
>>>>>>>> eate(Ref lectionServiceFactoryBean.java:242)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean.c
>>>>>>>> reate(Ja
>>>>>>>> xWsServi ceFactoryBean.java:202)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.apache.cxf.frontend.AbstractWSDLBasedEndpointFacto
>>>>>>>> ry.creat
>>>>>>>> eEndpoin t(AbstractWSDLBasedEndpointFactory.java:101)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.apache.cxf.frontend.ClientFactoryBean.create(Clien
>>>>>>>> tFactory
>>>>>>>> Bean.jav a:90)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.apache.cxf.frontend.ClientProxyFactoryBean.create(
>>>>>>>> ClientPr
>>>>>>>> oxyFacto ryBean.java:117)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.apache.cxf.jaxws.JaxWsProxyFactoryBean.create(JaxW
>>>>>>>> sProxyFa
>>>>>>>> ctoryBea n.java:124)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>>    sun.reflect.NativeMethodAccessorImpl.invoke0(
>>>>>>>>    Native
>>>>>>>>    Method)
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMeth
>>>>>>>> odAccess
>>>>>>>> orImpl.j ava:39)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(Delega
>>>>>>>> tingMeth
>>>>>>>> odAccess orImpl.java:25)
>>>>>>>> 
>>>>>>>>    at
>>>>>>>>    java.lang.reflect.Method.invoke(Method.java:5
>>>>>>>>    97)
>>>>>>>>    at
>>>>>>>> 
>>>>>>>> org.springframework.beans.factory.support.SimpleInstan
>>>>>>>> tiationS
>>>>>>>> trategy.
>>>>>>>> instantiate(SimpleInstantiationStrategy.java:145)
>>>>>>>> 
>>>>>>>>    ... 11 more
>>>>>>>> 
>>>>>>>> Are we not configuring our client correctly?  Just to
>>>>>>>> reiterate we are using 2.1.3.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Al
>>>>>>>> 
>>>>>>>> On Fri, Jul 22, 2011 at 5:09 PM, Daniel Kulp
>>>>>>>> <dkulp@apache.org>
>>>>> 
>>>>> wrote:
>>>>>>>>> On Friday, July 22, 2011 4:16:39 PM Algirdas Veitas wrote:
>>>>>>>>>> Small world :)
>>>>>>>>>> 
>>>>>>>>>> RE: Hmm... with #3, you shouldn't even be
>>>>>>>>>> hitting this.
>>>>>>>>>> 
>>>>>>>>>> The
>>>>>>>>> 
>>>>>>>>> HEADER_LIST
>>>>>>>>> 
>>>>>>>>>> should
>>>>>>>>>> be completely created per request since it
>>>>>>>>>> wouldn't be
>>>>>>>>>> pulled
>>>>>>>>>> from the request
>>>>>>>>>> context.     However, if the HEADER_LIST is
>>>>>>>>>> added to the
>>>>>>>>>> request
>>>>>>>>> 
>>>>>>>>> context  at
>>>>>>>>> 
>>>>>>>>>> ANY time, even using #3 after that would cause
>>>>>>>>>> an issue.
>>>>>>>>>> 
>>>>>>>>>> If you
>>>>>>>>>> 
>>>>>>>>>> aren't touching the HEADER_LIST anywhere, then
>>>>>>>>>> I'm not
>>>>>>>>>> really
>>>>>>>>>> sure what is
>>>>>>>>> 
>>>>>>>>> going
>>>>>>>>> 
>>>>>>>>>> on.
>>>>>>>>>> 
>>>>>>>>>> Yeah, we are not manipulating the HEADER_LIST
>>>>>>>>>> anywhere
>>>>>>>>>> in our
>>>>>>>>>> code, so
>>>>>>>>> 
>>>>>>>>> am
>>>>>>>>> 
>>>>>>>>>> not sure what is going on either....
>>>>>>>>>> 
>>>>>>>>>> RE: 2) After returning from any method where you
>>>>>>>>>> had set
>>>>>>>>>> a
>>>>>>>>>> header,
>>>>>>>>> 
>>>>>>>>> call:  *
>>>>>>>>> 
>>>>>>>>>> context*.getMessageContext().remove(Header.HEADE
>>>>>>>>>> R_LIST);
>>>>>>>>>> 
>>>>>>>>>> How do I get a handle to the "context", in our
>>>>>>>>>> situation
>>>>>>>>>> where
>>>>>>>>>> we are
>>>>>>>>> 
>>>>>>>>> using
>>>>>>>>> 
>>>>>>>>>> generated code?
>>>>>>>>> 
>>>>>>>>> Sorry.  That would be the actual proxy object.   
>>>>>>>>> For
>>>>>>>>> example:
>>>>>>>>> 
>>>>>>>>> MyServiceInterface proxy =
>>>>>>>>> service.getMyServicePort();
>>>>>>>>> ((BindingProvider)proxy).getRequestContext().....
>>>>>>>>> 
>>>>>>>>> With JAX-WS, all the proxys implement not only the
>>>>>>>>> generated
>>>>>>>>> interface,
>>>>>>>>> but
>>>>>>>>> also the BindingProvider interface.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Dan
>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> Al
>>>>>>>>>> 
>>>>>>>>>> On Fri, Jul 22, 2011 at 4:04 PM, Daniel Kulp
>>>>>>>>>> <dkulp@apache.org>
>>>>> 
>>>>> wrote:
>>>>>>>>>>> On Friday, July 22, 2011 3:58:06 PM Algirdas Veitas wrote:
>>>>>>>>>>>> Hi Daniel,
>>>>>>>>>>>> 
>>>>>>>>>>>> Nice meeting you as well!
>>>>>>>>>>> 
>>>>>>>>>>> Just discovered you and I went to
>>>>>>>>>>> Northeastern at
>>>>>>>>>>> roughly
>>>>>>>>>>> the same
>>>>>>>>> 
>>>>>>>>> time.
>>>>>>>>> 
>>>>>>>>>>> We
>>>>>>>>>>> may have met before.  :-)
>>>>>>>>>>> 
>>>>>>>>>>>> One follow up question....the way we are
>>>>>>>>>>>> adding
>>>>>>>>>>>> headers
>>>>>>>>>>>> is #3.  Our generated client code treats
>>>>>>>>>>>> our
>>>>>>>>>>>> headers
>>>>>>>>>>>> like an ordinary parameter in
>>>>>>>>>>> 
>>>>>>>>>>> the
>>>>>>>>>>> 
>>>>>>>>>>>> method signature.  Will your solution
>>>>>>>>>>>> work for
>>>>>>>>>>>> this
>>>>>>>>>>>> strategy as
>>>>>>>>>>>> well?
>>>>>>>>>>> 
>>>>>>>>>>> Hmm... with #3, you shouldn't even be
>>>>>>>>>>> hitting this.
>>>>>>>>>>> 
>>>>>>>>>>> The
>>>>>>>>>>> 
>>>>>>>>>>> HEADER_LIST should
>>>>>>>>>>> be completely created per request since it
>>>>>>>>>>> wouldn't
>>>>>>>>>>> be
>>>>>>>>>>> pulled from
>>>>>>>>> 
>>>>>>>>> the
>>>>>>>>> 
>>>>>>>>>>> request
>>>>>>>>>>> context.     However, if the HEADER_LIST is
>>>>>>>>>>> added to
>>>>>>>>>>> the
>>>>>>>>>>> request
>>>>>>>>> 
>>>>>>>>> context
>>>>>>>>> 
>>>>>>>>>>> at
>>>>>>>>>>> 
>>>>>>>>>>> ANY time, even using #3 after that would
>>>>>>>>>>> cause an
>>>>>>>>>>> issue.  If
>>>>>>>>>>> you
>>>>>>>>> 
>>>>>>>>> aren't
>>>>>>>>> 
>>>>>>>>>>> touching the HEADER_LIST anywhere, then I'm
>>>>>>>>>>> not
>>>>>>>>>>> really sure
>>>>>>>>>>> what is
>>>>>>>>>>> going
>>>>>>>>>>> on.
>>>>>>>>>>> Strange.   Definitely give it a try though.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Dan
>>>>>>>>>>> 
>>>>>>>>>>>> Al
>>>>>>>>>>>> 
>>>>>>>>>>>> On Fri, Jul 22, 2011 at 3:45 PM, Daniel
>>>>>>>>>>>> Kulp
>>>>>>>>>>>> <dkulp@apache.org>
>>>>>>>>> 
>>>>>>>>> wrote:
>>>>>>>>>>>>> We talked briefly about this at
>>>>>>>>>>>>> lunch today
>>>>>>>>>>>>> (nice to
>>>>>>>>>>>>> meet you
>>>>>>>>>>>>> Al!) but wanted
>>>>>>>>>>>>> to follow up here.....
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> The per-proxy request context can
>>>>>>>>>>>>> definitely
>>>>>>>>>>>>> come
>>>>>>>>>>>>> into play
>>>>>>>>>>>>> here.   If any of
>>>>>>>>>>>>> the threads add a header list to the
>>>>>>>>>>>>> RequestContext
>>>>>>>>>>>>> via the
>>>>>>>>>>>>> method
>>>>>>>>>>>>> mentioned
>>>>>>>>> 
>>>>>>>>>>>>> in #4 of:
>>>>>>>>> http://cxf.apache.org/faq#FAQ-HowcanIaddsoapheaderst
>>>>>>>>> otherequ
>>>>>>>>> est%2Fre spon
>>>>>>>>> 
>>>>>>>>>>>>> se%3F
>>>>>>>>>>>>> 
>>>>>>>>>>>>> then those headers would be sent on
>>>>>>>>>>>>> all
>>>>>>>>>>>>> methods
>>>>>>>>>>>>> called on the
>>>>>>>>>>>>> proxy.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> What's worse, I THINK that List is
>>>>>>>>>>>>> just
>>>>>>>>>>>>> copied into
>>>>>>>>>>>>> the real
>>>>>>>>>>>>> message
>>>>>>>>>>>>> context
>>>>>>>>>>>>> so any header processing done during
>>>>>>>>>>>>> the
>>>>>>>>>>>>> processing
>>>>>>>>>>>>> of the
>>>>>>>>>>>>> message
>>>>>>>>>>> 
>>>>>>>>>>> would
>>>>>>>>>>> 
>>>>>>>>>>>>> affect that list, thus affecting all
>>>>>>>>>>>>> threads
>>>>>>>>>>>>> as well
>>>>>>>>>>>>> as future
>>>>>>>>>>>>> method
>>>>>>>>>>>>> calls on
>>>>>>>>>>>>> the same thread.      Thinking about
>>>>>>>>>>>>> this,
>>>>>>>>>>>>> it may
>>>>>>>>>>>>> make sense to
>>>>>>>>>>>>> clone
>>>>>>>>>>>>> the
>>>>>>>>>>>>> list
>>>>>>>>>>>>> at the very start of processing
>>>>>>>>>>>>> messages to
>>>>>>>>>>>>> make
>>>>>>>>>>>>> sure the header list
>>>>>>>>>>> 
>>>>>>>>>>> in
>>>>>>>>>>> 
>>>>>>>>>>>>> the
>>>>>>>>>>>>> request context isn't modified.  
>>>>>>>>>>>>> That might
>>>>>>>>>>>>> be
>>>>>>>>>>>>> worth filing a
>>>>>>>>>>>>> jira
>>>>>>>>>>> 
>>>>>>>>>>> for.
>>>>>>>>>>> 
>>>>>>>>>>>>> For now, I would suggest doing 2
>>>>>>>>>>>>> things:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 1) Use the thread local request
>>>>>>>>>>>>> contexts:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> <jaxws:client id="xService"
>>>>>>>>>>>>> 
>>>>>>>>>>>>>        serviceClass="com.XS
>>>>>>>>>>>>>        ervice"
>>>>>>>>>>>>>        address="#xServiceUr
>>>>>>>>>>>>>        l" >
>>>>>>>>>>>>> 
>>>>>>>>>>>>>    <jaxws:properties>
>>>>>>>>>>>>> 
>>>>>>>>>>>>>           <entry
>>>>>>>>>>>>>           key="thread.lo
>>>>>>>>>>>>>           cal.requ
>>>>>>>>>>>>>           est.con
>>>>>>>>>>>>>           text"
>>>>>>>>>>>>>           value="true"
>>>>>>>>>>>>>           />
>>>>>>>>>>>>> 
>>>>>>>>>>>>>       </jaxws:properties>
>>>>>>>>>>>>> 
>>>>>>>>>>>>> </jaxws:client>
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 2) After returning from any method
>>>>>>>>>>>>> where you
>>>>>>>>>>>>> had set
>>>>>>>>>>>>> a header,
>>>>>>>>>>>>> call:
>>>>>>>>>>>>> context.getMessageContext().remove(H
>>>>>>>>>>>>> eader.HE
>>>>>>>>>>>>> ADER_LIS
>>>>>>>>>>>>> T);
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> That should keep things in a good
>>>>>>>>>>>>> state.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Dan
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Friday, July 22, 2011 10:25:10 AM
>>>>>>>>>>>>> Algirdas Veitas
>>> 
>>> wrote:
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> We are getting a
>>>>>>>>>>>>>> ConcurrentModificationException
>>>>>>>>>>>>>> exception
>>>>>>>>>>>>>> in our
>>>>>>>>>>>>>> CXF
>>>>>>>>>>>>>> client, when it is processing
>>>>>>>>>>>>>> SOAPHeaders.    We
>>>>>>>>>>>>>> did find
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>> following link that describes
>>>>>>>>>>>>>> what is
>>>>>>>>>>>>>> going on
>>>>>>>>> 
>>>>>>>>> http://cxf.547215.n5.nabble.com/jira-Created-CXF-276
>>>>>>>>> 2-Cannot
>>>>>>>>> -deploy- csta
>>>>>>>>> 
>>>>>>>>>>>>> -web
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> -service-td586662.html#a586663,
>>>>>>>>>>>>>> and we
>>>>>>>>>>>>>> will
>>>>>>>>>>>>>> address that
>>>>>>>>>>>>>> (Header
>>>>>>>>>>>>>> object
>>>>>>>>>>>>>> needs to be unique across
>>>>>>>>>>>>>> threads), but
>>>>>>>>>>>>>> something else
>>>>>>>>>>>>>> strange is
>>>>>>>>>>>>>> happening:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> To start here is our client
>>>>>>>>>>>>>> setup
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>  <!-- Web Service Clients -->
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>    <jaxws:client
>>>>>>>>>>>>>>    id="xService"
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>        serviceClass="co
>>>>>>>>>>>>>>        m.XServi
>>>>>>>>>>>>>>        ce"
>>>>>>>>>>>>>>        address="#xServi
>>>>>>>>>>>>>>        ceUrl"
>>>>>>>>>>>>>>        />
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> This WSDL that is associate with
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>> client has
>>>>>>>>>>>>>> some
>>>>>>>>>>>>>> methods that
>>>>>>>>>>>>>> DO
>>>>>>>>>>>>> 
>>>>>>>>>>>>> have a
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Header and some DO NOT.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> And here is the shortened stack
>>>>>>>>>>>>>> trace:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> <http://newportave.jira.com/wiki
>>>>>>>>>>>>>> /display
>>>>>>>>>>>>>> /21+Jul+
>>>>>>>>>>>>>> 2011+19/11%3
>>>>>>>>>>>>>> A46%2C87 4>
>>>>>>>>>>>>>> java.util.ConcurrentModification
>>>>>>>>>>>>>> Exceptio
>>>>>>>>>>>>>> n
>>>>>>>>>>>>>> at
>>>>>>>>> 
>>>>>>>>> java.util.AbstractList$Itr.checkForComodification(Ab
>>>>>>>>> stractLi
>>>>>>>>> st.java: 372)
>>>>>>>>> 
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> java.util.AbstractList$Itr.next(
>>>>>>>>>>>>>> Abstract
>>>>>>>>>>>>>> List.ja
>>>>>>>>>>>>>> va:343)
>>>>>>>>>>>>>> at
>>>>>>>>> 
>>>>>>>>> org.apache.cxf.binding.soap.interceptor.SoapOutInter
>>>>>>>>> ceptor.w
>>>>>>>>> riteSoap Enve
>>>>>>>>> 
>>>>>>>>>>>>> lope
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Start(SoapOutInterceptor.java:13
>>>>>>>>>>>>>> 9) at
>>>>>>>>> 
>>>>>>>>> org.apache.cxf.binding.soap.interceptor.SoapOutInter
>>>>>>>>> ceptor.h
>>>>>>>>> andleMes sage
>>>>>>>>> 
>>>>>>>>>>>>> (Soa
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> pOutInterceptor.java:81) at
>>>>>>>>> 
>>>>>>>>> org.apache.cxf.binding.soap.interceptor.SoapOutInter
>>>>>>>>> ceptor.h
>>>>>>>>> andleMes sage
>>>>>>>>> 
>>>>>>>>>>>>> (Soa
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> pOutInterceptor.java:61) at
>>>>>>>>> 
>>>>>>>>> org.apache.cxf.phase.PhaseInterceptorChain.doInterce
>>>>>>>>> pt(Phase
>>>>>>>>> Intercep torC
>>>>>>>>> 
>>>>>>>>>>>>> hain
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> .java:255) at
>>>>>>>>>>>>> 
>>>>>>>>>>>>> org.apache.cxf.endpoint.ClientImpl.i
>>>>>>>>>>>>> nvoke(Cl
>>>>>>>>>>>>> ientImpl
>>>>>>>>>>>>> .java:516)
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Here is the rub: This exception
>>>>>>>>>>>>>> is being
>>>>>>>>>>>>>> thrown
>>>>>>>>>>>>>> when we are
>>>>>>>>>>>>>> calling
>>>>>>>>>>>>>> a
>>>>>>>>>>>>> 
>>>>>>>>>>>>> method
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> that DOES NOT have a Header, but
>>>>>>>>>>>>>> on line
>>>>>>>>>>>>>> 139 in
>>>>>>>>>>>>>> SoapOutInterceptor,
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>> looks like we are trying to
>>>>>>>>>>>>>> process an
>>>>>>>>>>>>>> list that
>>>>>>>>>>>>>> has atleast
>>>>>>>>>>>>>> 1
>>>>>>>>>>>>>> header.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Am
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> pretty certain (this is
>>>>>>>>>>>>>> occurring only
>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>> specific
>>>>>>>>>>>>>> environment)
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> other methods on this service
>>>>>>>>>>>>>> were made
>>>>>>>>>>>>>> previous
>>>>>>>>>>>>>> to this
>>>>>>>>>>>>>> error that
>>>>>>>>>>>>>> did
>>>>>>>>>>>>>> include a SOAP Header.  Am
>>>>>>>>>>>>>> speculating,
>>>>>>>>>>>>>> but is
>>>>>>>>>>>>>> the SOAP
>>>>>>>>>>>>>> Header being
>>>>>>>>>>>>> 
>>>>>>>>>>>>> cached
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> in the request context and the
>>>>>>>>>>>>>> request
>>>>>>>>>>>>>> context
>>>>>>>>>>>>>> is scoped per
>>>>>>>>>>>>>> client
>>>>>>>>>>>>>> instance as per the FAQ (
>>>>>>>>>>>>>> http://cxf.apache.org/faq.html#F
>>>>>>>>>>>>>> AQ-AreJA
>>>>>>>>>>>>>> XWSclien
>>>>>>>>>>>>>> tproxiesthre
>>>>>>>>>>>>>> adsafe%3 F)?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Al
>>>>>>>>>>>>> 
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Daniel Kulp
>>>>>>>>>>>>> dkulp@apache.org
>>>>>>>>>>>>> http://dankulp.com/blog
>>>>>>>>>>>>> Talend - http://www.talend.com
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> Daniel Kulp
>>>>>>>>>>> dkulp@apache.org
>>>>>>>>>>> http://dankulp.com/blog
>>>>>>>>>>> Talend - http://www.talend.com
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Daniel Kulp
>>>>>>>>> dkulp@apache.org
>>>>>>>>> http://dankulp.com/blog
>>>>>>>>> Talend - http://www.talend.com
>>>>> 
>>>>> --
>>>>> Daniel Kulp
>>>>> dkulp@apache.org
>>>>> http://dankulp.com/blog
>>>>> Talend - http://www.talend.com
>>> 
>>> --
>>> Daniel Kulp
>>> dkulp@apache.org
>>> http://dankulp.com/blog
>>> Talend - http://www.talend.com
> -- 
> Daniel Kulp
> dkulp@apache.org
> http://dankulp.com/blog
> Talend - http://www.talend.com

Mime
View raw message