Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 69DB12009E2 for ; Wed, 1 Jun 2016 16:07:31 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 68085160A4C; Wed, 1 Jun 2016 14:07:31 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 196EA160A45 for ; Wed, 1 Jun 2016 16:07:29 +0200 (CEST) Received: (qmail 38840 invoked by uid 500); 1 Jun 2016 14:07:29 -0000 Mailing-List: contact users-help@qpid.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@qpid.apache.org Delivered-To: mailing list users@qpid.apache.org Received: (qmail 38829 invoked by uid 99); 1 Jun 2016 14:07:28 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Jun 2016 14:07:28 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 5F5A2C0DF1 for ; Wed, 1 Jun 2016 14:07:28 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.427 X-Spam-Level: X-Spam-Status: No, score=-1.427 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-1.426, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 4SMrJbdSM2Ad for ; Wed, 1 Jun 2016 14:07:24 +0000 (UTC) Received: from mail-gw-oslo.pgs.com (mail-gw-oslo.pgs.com [217.144.243.14]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 683905F4E3 for ; Wed, 1 Jun 2016 14:07:24 +0000 (UTC) Received: from pps.filterd (mail-gw-oslo.pgs.com [127.0.0.1]) by mail-gw-oslo.pgs.com (8.15.0.59/8.15.0.59) with SMTP id u5191iH0028653 for ; Wed, 1 Jun 2016 02:01:44 -0700 Received: from oslms27a.onshore.pgs.com (oslms27a.onshore.pgs.com [10.30.16.18]) by mail-gw-oslo.pgs.com with ESMTP id 238hsp9qv9-1 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 01 Jun 2016 02:01:42 -0700 Received: from [10.30.39.189] (10.30.39.189) by OSLMS27A.onshore.pgs.com (10.30.16.18) with Microsoft SMTP Server (TLS) id 14.3.266.1; Wed, 1 Jun 2016 16:07:15 +0200 Subject: Re: Lock-up in waitForCompletion() To: References: <5744215D.9080100@pgs.com> <57443901.2090605@redhat.com> <57443978.8000107@redhat.com> <57446500.9050808@pgs.com> <57446830.1070606@redhat.com> <5745A047.8040407@pgs.com> <57472146.9080609@redhat.com> <574E9303.9010009@pgs.com> <574E9C80.9020705@redhat.com> From: Toralf Lund Message-ID: <574EEC13.9070500@pgs.com> Date: Wed, 1 Jun 2016 16:07:15 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 MIME-Version: 1.0 In-Reply-To: <574E9C80.9020705@redhat.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.30.39.189] X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-06-01_06:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=13 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1604210000 definitions=main-1606010165 archived-at: Wed, 01 Jun 2016 14:07:31 -0000 On 01/06/16 10:27, Gordon Sim wrote: > On 01/06/16 08:47, Toralf Lund wrote: >> Hi, >> >> I haven't had time to look at this again until now... >> >> I restarted qpidd the on the (current) backup server, and after that, I >> can't reproduce the issue. I guess this means it was indeed related to >> replication. Unfortunately, I never managed to enable protocol trace so >> I could verify it. How is it done, again? (Please excuse my >> forgetfulness.) I tried setting QPID_LOG_ENABLE to "trace:Protocol" >> before starting the process, but it produced no output. > > For AMQP 0-10 you need to use trace:Network (or trace+:Network). Of course, I knew that, really :-/ It turns out I saved "full" trace data at one stage, but I think perhaps this was after the restart. I've included the initial "[Network]" lines at the end of this message in case it contains something funny. > >> Maybe I might >> have done something on the broker instead, but turning on trace for >> everything didn't seem like a good idea... >> >> Some questions that remain to be answered: >> >> 1. If this was a backup broker issue, why did it affect only one queue? >> 2. Is there any way to configure the broker(s) or queues so that there >> won't be a complete lock-up like this? I'd rather loose the >> replication if there is an issue, but I don't really want to disable >> it altogether, if you know what I'm saying... > > Were there any error messages in the logs of either the primary or the > backups? I can't find anything in qpidd.log on any of the hosts. 2016-06-01 09:15:17 [Network] debug TCPConnector created for 0-10 2016-06-01 09:15:17 [Network] debug RECV [[10.30.39.189:43270-cirm:5672]]: INIT(0-10) 2016-06-01 09:15:17 [Network] trace RECV [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=0; {ConnectionStartBody: server-properties={host:V2:9:str16(lys-cirm2),platform:V2:5:str16(Linux),product:V2:8:str16(qpid-cpp),qpid.federation_tag:V2:36:str16(587df410-04d8-42a8-8ac4-89e3b058f951),version:V2:4:str16(0.34)}; mechanisms=str16{V2:9:str16(ANONYMOUS), V2:5:str16(PLAIN)}; locales=str16{V2:5:str16(en_US)}; }] 2016-06-01 09:15:17 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=0; {ConnectionStartOkBody: client-properties={qpid.client_pid:F4:int32(2983),qpid.client_ppid:F4:int32(1577),qpid.client_process:V2:11:str16(COMRADEbocs),qpid.session_flow:F4:int32(1)}; mechanism=ANONYMOUS; response=xxxxxx; locale=en_US; }] 2016-06-01 09:15:17 [Network] trace RECV [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=0; {ConnectionTuneBody: channel-max=65534; max-frame-size=65535; heartbeat-min=0; heartbeat-max=120; }] 2016-06-01 09:15:17 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=0; {ConnectionTuneOkBody: channel-max=32767; max-frame-size=65535; heartbeat=2; }] 2016-06-01 09:15:17 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=0; {ConnectionOpenBody: virtual-host=; capabilities=void{}; insist=1; }] 2016-06-01 09:15:17 [Network] trace RECV [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=0; {ConnectionOpenOkBody: known-hosts=str16{V2:26:str16(amqp:tcp:10.30.15.215:5672)}; }] 2016-06-01 09:15:17 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {SessionAttachBody: name=8bbe0e18-0941-4ac7-85a2-adf90ffa5449; }] 2016-06-01 09:15:17 [Network] trace RECV [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {SessionAttachedBody: name=8bbe0e18-0941-4ac7-85a2-adf90ffa5449; }] 2016-06-01 09:15:17 [Network] trace RECV [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {SessionCommandPointBody: command-id=0; command-offset=0; }] 2016-06-01 09:15:17 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {SessionRequestTimeoutBody: timeout=0; }] 2016-06-01 09:15:17 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {SessionCommandPointBody: command-id=0; command-offset=0; }] 2016-06-01 09:15:17 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {ExecutionSyncBody: }] 2016-06-01 09:15:17 [Network] trace RECV [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {SessionTimeoutBody: timeout=0; }] 2016-06-01 09:15:17 [Network] trace RECV [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {SessionCompletedBody: commands={ [0,0] }; }] 2016-06-01 09:15:17 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {ExchangeDeclareBody: exchange=DATA.topic.exch; type=topic; alternate-exchange=; arguments={}; }] 2016-06-01 09:15:17 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {ExecutionSyncBody: }] 2016-06-01 09:15:17 [Network] trace RECV [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {SessionCompletedBody: commands={ [0,2] }; }] 2016-06-01 09:15:17 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:17 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (182 bytes); properties={{MessageProperties: content-length=49; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:21:vbin16(BOCS.Status.Heartbeat),qpid.subject:V2:21:str16(bocs.status.heartbeat),timestamp:F8:int64(1464765315)}; }{DeliveryProperties: ttl=2147483647; routing-key=bocs.status.heartbeat; }}] 2016-06-01 09:15:17 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (49 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=31; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.0.0),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.0.0; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (31 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=29; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.0.1),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.0.1; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (29 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=37; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.0.2),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.0.2; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (37 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=36; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.1.0),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.0; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (36 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.1.1),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.1; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.1.2),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.2; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.1.3),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.3; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.1.4),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.4; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.1.5),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.5; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.1.6),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.6; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.1.7),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.7; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {SessionFlushBody: completed=1; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.1.8),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.8; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace RECV [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {SessionCompletedBody: commands={ [0,6] }; timely-reply=1; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {SessionKnownCompletedBody: commands={ [0,6] }; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (173 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:22:str16(bocs.status.config.1.9),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.9; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (175 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:23:str16(bocs.status.config.1.10),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.10; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (175 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:23:str16(bocs.status.config.1.11),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.11; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (175 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:23:str16(bocs.status.config.1.12),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.12; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (175 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:23:str16(bocs.status.config.1.13),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.13; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (175 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:23:str16(bocs.status.config.1.14),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.14; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (175 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:23:str16(bocs.status.config.1.15),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.15; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (175 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:23:str16(bocs.status.config.1.16),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.16; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (175 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:23:str16(bocs.status.config.1.17),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.17; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (175 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:23:str16(bocs.status.config.1.18),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.18; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (175 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:23:str16(bocs.status.config.1.19),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.19; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[be; channel=1; header (175 bytes); properties={{MessageProperties: content-length=32; correlation-id=; content-type=application/x-protobuf; user-id=; application-headers={message_type:V2:18:vbin16(BOCS.Status.Config),qpid.subject:V2:23:str16(bocs.status.config.1.20),timestamp:F8:int64(1464340014)}; }{DeliveryProperties: routing-key=bocs.status.config.1.20; }}] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Ebe; channel=1; content (32 bytes) 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[BEbe; channel=1; {SessionFlushBody: completed=1; }] 2016-06-01 09:15:18 [Network] trace SENT [[10.30.39.189:43270-cirm:5672]]: Frame[Bbe; channel=1; {MessageTransferBody: destination=DATA.topic.exch; accept-mode=1; acquire-mode=0; }] --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For additional commands, e-mail: users-help@qpid.apache.org