qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajith Attapattu <rajit...@gmail.com>
Subject Re: Qpid Cluster Errors
Date Wed, 16 Dec 2009 00:24:17 GMT
On Tue, Dec 15, 2009 at 6:40 PM, Sandy Pratt <prattrs@adobe.com> wrote:
> Rajith,
>
> I'll see what I can do.  It will take a little bit to make sure there are no internal
dependencies (libraries and maven repos are what I can think of now) but it should be doable.
>
> I do have sync_publish='all' set in my connection URL.  I was under the impression that
this was a synonym for sycn_ack.  I'll do some testing on that.
no, sync_publish will dictate which transfers (transient, durable or
all) should be done synchronously.
sync_ack makes the client consume each message synchronously.

I'd be interested to know
a) if you could reproduce the issue faster if you used sync_ack
b) Whether sync_publish has any affect at all - (I think it does not).
c) Are you using a message listener or are you using the receive() method?
c) Your actual code, so I can compare it with my test client to see if
there are any differences and experiment based on that.

Btw thanks for your interest on this. Your test case illustrates that
we still have a few corner cases that haven't been identified yet.

> Thanks,
>
> Sandy
>
>> -----Original Message-----
>> From: Rajith Attapattu [mailto:rajith77@gmail.com]
>> Sent: Tuesday, December 15, 2009 3:29 PM
>> To: users@qpid.apache.org
>> Subject: Re: Qpid Cluster Errors
>>
>> Sandy,
>>
>> Interesting test case.
>> I reproduced a similar issue (well atleast the symptoms were the same)
>> but apparently it looks like it's different from your issue.
>> Could you please open a JIRA and submit your test code?
>> I want to see how your test case is different from the one I have.
>> I was able to reproduce my issue reliably and quickly by  using
>> sync_ack=true in the connection URL.
>>
>> I also want to see how I could add your test case to the soak/stress
>> test suite I am building under the testkit module.
>>
>> Regards,
>>
>> Rajith
>>
>> On Tue, Dec 15, 2009 at 4:32 PM, Sandy Pratt <prattrs@adobe.com> wrote:
>> >> > -----Original Message-----
>> >> > From: Alan Conway [mailto:aconway@redhat.com]
>> >> >
>> >> > There is another issue that turned up with the same symptom
>> >> > https://issues.apache.org/jira/browse/QPID-225. Fixed in r888874.
>> Let
>> >> > me know if
>> >> > that resolves your problem, if not I want to look into it more.
>> >>
>> >> Thanks Alan.
>> >>
>> >> I don't think that's the issue, because this system should only be
>> >> receiving connections from clients who are using AUTO_ACKNOWLEDGE,
>> but
>> >> I'm happy to try it.
>> >>
>> >> I will let you know how it goes.
>> >
>> > I made a new build of the java client from trunk today, and used that
>> to spam a cluster of two brokers running qpidd-0.5.752581-34 and rhm-
>> 0.5.3206-27 and saw the error again:
>> >
>> > 2009-dec-15 12:10:34 trace 10.59.174.211:19320(READY) DLVR 27677:
>> Frame[BEbe; channel=0; {SessionCompletedBody: commands={ [0,203] }; }]
>> data 10.59.174.186:24880-40
>> > 2009-dec-15 12:10:34 debug prattrs@QPID.79d87e4e-1d7c-4c1c-84f2-
>> 984aeb8c66c2: sender marked completed: { [0,203] }
>> > 2009-dec-15 12:10:34 debug Exception constructed:
>> prattrs@QPID.79d87e4e-1d7c-4c1c-84f2-984aeb8c66c2: confirmed < (204+0)
>> but only sent < (203+0) (qpid/SessionState.cpp:163)
>> > 2009-dec-15 12:10:34 error Execution exception: invalid-argument:
>> prattrs@QPID.79d87e4e-1d7c-4c1c-84f2-984aeb8c66c2: confirmed < (204+0)
>> but only sent < (203+0) (qpid/SessionState.cpp:163)
>> > 2009-dec-15 12:10:34 error 10.59.174.211:19320(READY/error) channel
>> error 27677 on 10.59.174.186:24880-40(shadow): invalid-argument:
>> prattrs@QPID.79d87e4e-1d7c-4c1c-84f2-984aeb8c66c2: confirmed < (204+0)
>> but only sent < (203+0) (qpid/SessionState.cpp:163) (unresolved:
>> 10.59.174.186:24880 10.59.174.211:19320 )
>> > 2009-dec-15 12:10:34 trace MCAST 10.59.174.211:19320-0:
>> {ClusterErrorCheckBody: type=1; frame-seq=27677; }
>> >
>> > OS is an up to date installation of RHEL 5.4 32 bit running on
>> VMWare.
>> >
>> > Qpidd.conf is:
>> >
>> > no-module-dir=true
>> > load-module=/usr/lib/qpid/daemon/cluster.so
>> > load-module=/usr/lib/qpid/daemon/msgstore.so
>> > cluster-mechanism=PLAIN
>> > cluster-username="***"
>> > cluster-password="***"
>> > cluster-name="na1m-dev1"
>> > log-to-file=/var/lib/qpidd/qpidd.log
>> > trace=1
>> > auth=yes
>> > wait=60
>> >
>> > JNDI connection string:
>> >
>> > connectionfactory.qpidConnectionFactory =
>> amqp://***:***@test/?brokerlist='tcp://***.com:443?ssl='true',connectde
>> lay='1000',connecttimeout='5000'',failover='roundrobin?cyclecount='999'
>> ',sync_publish='all'
>> >
>> > The SSL is terminated at the load balancer.
>> >
>> > I'd love to hear the community's ideas on where to look next.  I
>> recently tested out a build of the 0.6beta1 release, and found that the
>> bug (or at least the same symptoms) occurred there as well.  I don't
>> have a good fast way to reproduce the problem.  This particular crash
>> occurred after I spammed the cluster for about 7 minutes, during which
>> time the spammer sent about 20k messages of between 1-1000 KB in size
>> through the cluster, whilst periodically closing and reopening sending
>> and receiving connections.
>> >
>> > Thanks!
>> >
>> > Sandy
>> >
>> > ---------------------------------------------------------------------
>> > Apache Qpid - AMQP Messaging Implementation
>> > Project:      http://qpid.apache.org
>> > Use/Interact: mailto:users-subscribe@qpid.apache.org
>> >
>> >
>>
>>
>>
>> --
>> Regards,
>>
>> Rajith Attapattu
>> Red Hat
>> http://rajith.2rlabs.com/
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project:      http://qpid.apache.org
>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project:      http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>



-- 
Regards,

Rajith Attapattu
Red Hat
http://rajith.2rlabs.com/

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


Mime
View raw message