Mailing-List: contact users-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@activemq.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1084)
Subject: Re: Random persistent messages lost
From: Alistair Young <alistair.young@uhi.ac.uk>
In-Reply-To: 
 <CAH+vQmOdWJwQ2RCcqN-H2tJ9KMVxx8hSOjKmKj=2-dAYthOakA@mail.gmail.com>
Date: Mon, 26 Sep 2011 16:19:02 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <A46AD839-7C2C-4A33-ABDF-A49B48F2CC1C@uhi.ac.uk>
References: <FD79E2E2-27D7-4403-A5FC-398901CDAA57@uhi.ac.uk>
 <93139781-4FDA-4E23-A8C5-BC71CCEB0745@fusesource.com>
 <4AA5AAA3-6442-426B-85F8-F45B5AF2D8DB@uhi.ac.uk>
 <3D2048DE-C0D3-4940-B1BF-FBE3EB0FF970@fusesource.com>
 <3594DF15-CA52-4A2D-9467-E88E82BFD560@uhi.ac.uk>
 <BEEEE766-6964-419E-99F4-2798211BCA4D@uhi.ac.uk>
 <CAH+vQmMzuQb6-==MystO2ipi+_-UgyxgXQXuz0MY2jGVdX8iMg@mail.gmail.com>
 <BC8724CF-A082-4350-BA8F-FE71EA8C836B@uhi.ac.uk>
 <F6BEA126-5C9E-482E-94B3-94DB9116A288@uhi.ac.uk>
 <B4210452-71D7-4F40-9BA2-20E3872BD31B@fusesource.com>
 <586FCDE6-B2B6-45BE-81DE-5A610E504B17@uhi.ac.uk>
 <37D4C3CA-E66F-433C-8D2E-7D0251E722C1@uhi.ac.uk>
 <95BBC082-B9FC-4CDD-8AE2-D2173550F90A@uhi.ac.uk>
 <CAH+vQmOdWJwQ2RCcqN-H2tJ9KMVxx8hSOjKmKj=2-dAYthOakA@mail.gmail.com>
To: users@activemq.apache.org

trying to log incoming messages at the broker to narrow the problem and =
another problem appears. This works on one server but adds 0x0 chars on =
another so the message fails to parse:

public boolean isAllowedToConsume(ConnectionContext context, Message =
message) {

new String(message.getContent().data, message.getContent().offset, =
message.getContent().length)

so on one server the offset and length are wrong as it seems to report =
too many chars

Alistair

--=20
mov eax,1
mov ebx,0
int 80h


On 22 Sep 2011, at 12:05, Gary Tully wrote:

> Is this code shared by multiple threads? does it need synchronization?
>=20
> What you are experiencing does seem odd, it would be great if you
> could provide a simple junit test case that can reproduce.
>=20
> Also, peeking that the code of the logging plugin, it should be
> logging sends, that is again odd:
> =
http://svn.apache.org/viewvc/activemq/tags/activemq-5.5.0/activemq-core/sr=
c/main/java/org/apache/activemq/broker/util/LoggingBrokerPlugin.java?view=3D=
markup
>=20
> On 22 September 2011 10:59, Alistair Young <alistair.young@uhi.ac.uk> =
wrote:
>> would someone be kind enough to have a quick look at this code to see =
if it could cause problems? The method returns a reusable connection so =
the user of the class doesn't have to connect/disconnect on every =
message. The first message never fails, it's always when reusing the =
connection that random messages disappear.
>>=20
>> thanks,
>>=20
>> Alistair
>>=20
>>    try {
>>      // Try to reuse a previous connection
>>      if (previousMxConnection =3D=3D null) {
>>        ActiveMQConnectionFactory connectionFactory =3D new =
ActiveMQConnectionFactory(brokerURL);
>>        connection =3D connectionFactory.createConnection();
>>        connection.setClientID(clientId);
>>        connection.start();
>>        session =3D connection.createSession(false, =
Session.AUTO_ACKNOWLEDGE);
>>        destination =3D session.createTopic(topicName);
>>        producer =3D session.createProducer(destination);
>>        producer.setDeliveryMode(messageMode);
>>=20
>>        mxConnection =3D new MatrixConnection();
>>        mxConnection.connection =3D connection;
>>        mxConnection.session =3D session;
>>        mxConnection.destination =3D destination;
>>        mxConnection.producer =3D producer;
>>      }
>>      else {
>>        session =3D previousMxConnection.session;
>>        if (previousMxConnection.producer =3D=3D null) {
>>          producer =3D =
session.createProducer(previousMxConnection.destination);
>>          producer.setDeliveryMode(DeliveryMode.PERSISTENT);
>>          previousMxConnection.producer =3D producer;
>>        }
>>        else {
>>          producer =3D previousMxConnection.producer;
>>        }
>>        mxConnection =3D previousMxConnection;
>>      }
>>=20
>>      TextMessage message =3D session.createTextMessage(messageText);
>>      producer.send(message);
>>=20
>>      return mxConnection;
>>    }
>>    catch(Exception e) {
>>      throw new MatrixClientException(e);
>>    }
>>  }
>>=20
>>=20
>> --
>> mov eax,1
>> mov ebx,0
>> int 80h
>>=20
>>=20
>>=20
>>=20
>> On 22 Sep 2011, at 10:50, Alistair Young wrote:
>>=20
>>> nothing seems to help. There are still messages just disappearing. =
Also the logging doesn't seem to log anything to do with producers:
>>>=20
>>> <loggingBrokerPlugin logAll=3D"true" logConnectionEvents=3D"true" />
>>>=20
>>> I can see lots of camel consumers from the local machine looking at =
the routes but nothing about incoming messages from producers.
>>>=20
>>> Alistair
>>>=20
>>> --
>>> mov eax,1
>>> mov ebx,0
>>> int 80h
>>>=20
>>>=20
>>>=20
>>>=20
>>> On 21 Sep 2011, at 13:13, Alistair Young wrote:
>>>=20
>>>> I'm running ActiveMQ 5.5.0 as a spring webapp rather than the =
standalone version. Would you recommend the standalone one? Not sure how =
to use camel with that though.
>>>>=20
>>>> there's quite a high rate of packet loss on the network it seems =
and according to this:
>>>> http://activemq.apache.org/async-sends.html
>>>> the default is sync send for persistent messages outside =
transactions. I'm using transactions on the broker but not the producer =
to preserve camel routes across restarts. So I presume the producer is =
using sync mode to send the messages as they are persistent. However, no =
error from the broker and no message arriving might hint at async mode =
falling foul of packet loss perhaps?
>>>>=20
>>>> I'll try this on the broker connection to see if it helps:
>>>>=20
>>>> jms.useAsyncSend=3Dfalse
>>>>=20
>>>> Alistair
>>>>=20
>>>>=20
>>>> --
>>>> mov eax,1
>>>> mov ebx,0
>>>> int 80h
>>>>=20
>>>>=20
>>>>=20
>>>>=20
>>>> On 21 Sep 2011, at 13:02, Torsten Mielke wrote:
>>>>=20
>>>>> What is the exact broker version used on your side?
>>>>> In case its not the latest released version, can you try the =
latest version?
>>>>>=20
>>>>> Do you set any particular headers on the Topic message in your =
producer?
>>>>>=20
>>>>>=20
>>>>>=20
>>>>> On Sep 21, 2011, at 1:54 PM, Alistair Young wrote:
>>>>>=20
>>>>>> I can't explain this at all. It's almost like the opposite of =
reliable messaging. At times, almost 1 in 3 messages just disappears. No =
errors. The KahaDB/db-*.log show no record of the message every arriving =
and yet the producer doesn't get an error.
>>>>>>=20
>>>>>> Alistair
>>>>>>=20
>>>>>>=20
>>>>>> --
>>>>>> mov eax,1
>>>>>> mov ebx,0
>>>>>> int 80h
>>>>=20
>>>=20
>>=20
>>=20
>=20
>=20
>=20
> --=20
> http://fusesource.com
> http://blog.garytully.com