Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 55F25200BD5 for ; Thu, 8 Dec 2016 17:41:42 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 548E9160B1F; Thu, 8 Dec 2016 16:41:42 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 52462160B0A for ; Thu, 8 Dec 2016 17:41:41 +0100 (CET) Received: (qmail 42896 invoked by uid 500); 8 Dec 2016 16:41:35 -0000 Mailing-List: contact users-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@nifi.apache.org Delivered-To: mailing list users@nifi.apache.org Received: (qmail 42886 invoked by uid 99); 8 Dec 2016 16:41:35 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Dec 2016 16:41:35 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 0D4361A953B for ; Thu, 8 Dec 2016 16:41:35 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.431 X-Spam-Level: * X-Spam-Status: No, score=1.431 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id TJTGGkFWkXk9 for ; Thu, 8 Dec 2016 16:41:33 +0000 (UTC) Received: from mail-oi0-f46.google.com (mail-oi0-f46.google.com [209.85.218.46]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id D5CAA60E3B for ; Thu, 8 Dec 2016 16:41:32 +0000 (UTC) Received: by mail-oi0-f46.google.com with SMTP id w63so458461681oiw.0 for ; Thu, 08 Dec 2016 08:41:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=LCyE1T3TJuDOXqEeYcppP1KDOXDEzSGW23im6701ebo=; b=eC61zG067ILdC9JPzuJXfpVIIiBgPYjdDDWFCTZX/rz3E0pBp+x3q9wsbNk7jVvea9 Ort8bZST78Ho7omk3i+BOePLM2peWji0DdVt3Dlv5mt64vOqsqy/JCq9ywPEg4s5zjhs DUTzvSUzC7ZQnZucHPNs47DYE7ACfyZG+MH+Bc3z3c3pVp1q6DTFVELEZhzW3RoxvllG 19yANikX0BXC+Bsmb6MSFw7bTMWqSGvSaQHMytdKJz9Z312iB8FXaDDUkhTqicUZ6h7J 067LWHoXkn/eXeyjUF39V+OSpVTcgR8rbfnfIvdQhhX+sMqqhOV63v8s3xsWb151rn/j YcZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=LCyE1T3TJuDOXqEeYcppP1KDOXDEzSGW23im6701ebo=; b=GvuAvMWucKQKvniqQOF1H99zJe2U/yrZcLyOh5a0wUpvgSLE/dJzPElJ2RWBOSAcC6 Fqo7KwMMyY8qxoWiB+wG7EODdb5/GHPcan6ee0qBwxzs1D0ZyS9p2nehhOWF7ljs451w F1O3xMNO3u5/ZeZrb72mNUjKjXFrjKAXL5GfZpr/AjPPc4po/3n4GPTOn+tzGM7K4R8h eYTaAtc+GZ21xMFqzEnp8pCuuiUpEdj8VIZZj1hmul7Ou3m18rDWMp30ws6jUiGWT/TW u0HuBUZTKr1a/zK0HugYb+2Z01zVI2QQiCHvuorUuSOZHWh8jk00Uw2OIlDw1IUlVRj2 VTuw== X-Gm-Message-State: AKaTC00mYXq0SbOyHp8+3stypQ1AgpFbTkSMBPtXjtsDMWGQjBS8YobpVGwvwhXPbyg3PAw9qCmPpCSlsmycqA== X-Received: by 10.202.78.76 with SMTP id c73mr38532119oib.65.1481215276618; Thu, 08 Dec 2016 08:41:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.3.242 with HTTP; Thu, 8 Dec 2016 08:41:16 -0800 (PST) In-Reply-To: <3889007F-4205-445F-8079-776AE17BB425@hortonworks.com> References: <9FC92D57-0F4B-44D6-9EAC-2B86FEF49793@hortonworks.com> <40CA29EC-06D1-40D9-AFBA-2F9131591C93@hortonworks.com> <3889007F-4205-445F-8079-776AE17BB425@hortonworks.com> From: James McMahon Date: Thu, 8 Dec 2016 11:41:16 -0500 Message-ID: Subject: Re: Nothing in rabbitMQ queue from Publish AMQP processor To: users@nifi.apache.org Content-Type: multipart/alternative; boundary=001a11c15902661a530543284eb4 archived-at: Thu, 08 Dec 2016 16:41:42 -0000 --001a11c15902661a530543284eb4 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Two quick updates. I generated a test message in the RMQ exchange using the RabbitMQ Management Console. My goal was to see if the Idle and blocking were problems or expected, and whether those would change when I sent a test message to the queue. It's also a good validation that I've bound the exchange and queue with a legit routing key. This worked - the test message was delivered successfully. I noticed that for each AMQP message nifi attempts to send to RMQ, RabbitMQ Management Console creates a Connection that is shown with State of blocking. and that is how it remains indefinitely. Nothing shows up in the exchange or in the queue. Jim On Thu, Dec 8, 2016 at 8:26 AM, Oleg Zhurakousky < ozhurakousky@hortonworks.com> wrote: > Fair enough > > On Dec 8, 2016, at 8:16 AM, James McMahon wrote: > > I'm a captive audience in this respect. I do not have the latitude to > change that, as it is dictated by a parent organization much broader than > my level. > > On Thu, Dec 8, 2016 at 8:10 AM, Oleg Zhurakousky < > ozhurakousky@hortonworks.com> wrote: > >> James >> >> Could you also see if you can upgrade to a newer version of NiFi? Curren= t >> release is 1.1.0, so 0.6.1 is about 3 releases behind and there were qui= te >> a few bug fixes and improvements since then that could shed some more li= ght >> into your issue. >> Meanwhile I=E2=80=99ll see what I can dig up on my end. >> >> Cheers >> Oleg >> >> On Dec 8, 2016, at 8:00 AM, James McMahon wrote: >> >> Yes absolutely Oleg. I will try to figure out how to make my rabbitMQ >> queues explicitly active. I'll need to dig into that because it is not >> plainly obvious to me how / if that needs to be done. I'll dig for that. >> >> I am using NiFi 0.6.1. >> >> Cheers and thanks again, >> >> Jim >> >> On Thu, Dec 8, 2016 at 7:46 AM, Oleg Zhurakousky < >> ozhurakousky@hortonworks.com> wrote: >> >>> James I have not used AMQP in a while (although I am the one responsibl= e >>> for developing the functionality you=E2=80=99re having problem with ;))= , so I need >>> to brush up a bit on some of the administrative actions you describe (i= dle >>> to active). Do i=E2=80=99ll get back to you, but as a joint debug step,= can you >>> please try to make your queens explicitly active and see if behavior >>> changes? >>> >>> Also, what version of NiFi you are using? >>> Cheers >>> Oleg >>> >>> On Dec 8, 2016, at 7:37 AM, James McMahon wrote: >>> >>> Good morning and thank you Oleg. No, no other stack traces or errors ar= e >>> indicated. I've got my Publish AMQP configuration set to DEBUG for Bull= etin >>> Level. >>> >>> I do get an error at the UI when I try to Start the Publish AMQP step >>> after I have Stopped it. It tells me "PublishAMQP[id=3D47f88.....744] c= annot >>> be started because it is not stopped. Current state is STOPPING". I sus= pect >>> this is a reflection of the WARN log message. >>> >>> In your experience must I take some action at the RabbitMQ Management >>> Console to change my Exchange and Queue from Idle to Active? I believe = that >>> would happen automatically when AMQP messages starting hitting the >>> exchange, but I thought it best to ask. I see nothing on the management >>> console that alludes to enabling the exchange or the queue. >>> >>> Jim >>> >>> On Thu, Dec 8, 2016 at 7:24 AM, Oleg Zhurakousky < >>> ozhurakousky@hortonworks.com> wrote: >>> >>>> James >>>> >>>> Aside from =E2=80=9Cclosing=E2=80=9D failure message, do you see any o= ther errors or >>>> stack traces in the logs? >>>> Meanwhile I=E2=80=99ll try to dig more with what I have? >>>> >>>> Oleg >>>> >>>> On Dec 8, 2016, at 7:13 AM, James McMahon wrote= : >>>> >>>> Hello I am trying to use a Publish AMQP step in my NiFi workflow. I >>>> generate a flow file with random text content using Generate Flow File= , and >>>> then send to a rabbitMQ Exchange named nifiAbcX using Publish AMQP. My >>>> exchange is bound to queue AbcQ by key **.Abc.**, and my virtual host >>>> is nifiVhost. I have established my exchange as a Topic type. >>>> >>>> >>>> My NiFi instance and my RabbitMQ instance are co-located for >>>> development test purposes on the same server. >>>> >>>> >>>> It appears that my Publish AMQP does indeed generate output which it >>>> sends down the "success" output flow path, but at the RabbitMQ Managem= ent >>>> Console I find that my Exchange and my Queue are Idle and show no mess= ages. >>>> My nifi log shows this Warning message: Failure while closing target >>>> resource AMQPublisher:amqp://test@127.0.0.1:5673/nifiVhost, >>>> EXCHANGE:nifiAbcX, ROUTING_KEY:xyz.Abc.prod What is this telling me? >>>> Why am I seeing no messages posted to my rabbitMQ queue through the >>>> exchange? Thank you in advance. >>>> >>>> >>>> >>> >>> >> >> > > --001a11c15902661a530543284eb4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Two quick updates. I generated a test message in the = RMQ exchange using the RabbitMQ Management Console. My goal was to see if t= he Idle and blocking were problems or expected, and whether those would cha= nge when I sent a test message to the queue. It's also a good validatio= n that I've bound the exchange and queue with a legit routing key. This= worked - the test message was delivered successfully.

=
I noticed that for each AMQP message nifi attempts to send to RMQ, Rab= bitMQ Management Console creates a Connection that is shown with State of b= locking. and that is=C2=A0how it remains indefinitely. Nothing shows up in = the exchange or in the queue.

Jim

On Thu, Dec 8, 2016 at= 8:26 AM, Oleg Zhurakousky <ozhurakousky@hortonworks.com>= ; wrote:
Fair enough

On Dec 8, 2016, at 8:16 AM, James McMahon <jsmcmahon3@gmail.com> wrote:

I'm a captive audience in this respect. I do not have = the latitude to change that, as it is dictated by a parent organization muc= h broader than my level.

On Thu, Dec 8, 2016 at 8:10 AM, Oleg Zhurakousky= <ozhur= akousky@hortonworks.com> wrote:
James

Could you also see if you can upgrade to a newer version of NiFi? Curr= ent release is 1.1.0, so 0.6.1 is about 3 releases behind and there were qu= ite a few bug fixes and improvements since then that could shed some more l= ight into your issue.
Meanwhile I=E2=80=99ll see what I can dig up on my end.

Cheers
Oleg

On Dec 8, 2016, at 8:00 AM, James McMahon <jsmcmahon3@gmail.com> wrote:

Yes absolutely Oleg. I will try to figure out how to make my rabbitMQ = queues explicitly active. I'll need to dig into that because it is not = plainly obvious to me how / if that needs to be done. I'll dig for that= .

I am using NiFi 0.6.1.

Cheers and thanks again,

Jim

On Thu, Dec 8, 2016 at 7:46 AM, Oleg Zhurakousky= <ozhur= akousky@hortonworks.com> wrote:
James I have not used AMQP in a while (although I am the one responsib= le for developing the functionality you=E2=80=99re having problem with ;)),= so I need to brush up a bit on some of the administrative actions you desc= ribe (idle to active). Do i=E2=80=99ll get back to you, but as a joint debug step, can you please try to make you= r queens explicitly active and see if behavior changes?

Also, what version of NiFi you are using?
Cheers
Oleg

On Dec 8, 2016, at 7:37 AM, James McMahon <jsmcmahon3@gmail.com> wrote:

Good morning and thank you Oleg. No, no other stack traces or errors a= re indicated. I've got my Publish AMQP configuration set to DEBUG for B= ulletin Level.

I do get an error at the UI=C2=A0when I try to Start the Publish AMQP = step after I have Stopped it. It tells me "PublishAMQP[id=3D47f88.....= 744] cannot be started because it is not stopped. Current state is STOPPING= ". I suspect this is a reflection of the WARN log message.

In your experience must I take some action at the RabbitMQ Management = Console to change my Exchange and Queue from Idle to Active? I believe that= would happen automatically when AMQP messages starting hitting the exchang= e, but I thought it best to ask. I see nothing on the management console that alludes to enabling t= he exchange or the queue.

Jim

On Thu, Dec 8, 2016 at 7:24 AM, Oleg Zhurakousky= <ozhur= akousky@hortonworks.com> wrote:
James

Aside from =E2=80=9Cclosing=E2=80=9D failure message, do you see any o= ther errors or stack traces in the logs?=C2=A0
Meanwhile I=E2=80=99ll try to dig more with what I have?

Oleg

On Dec 8, 2016, at 7:13 AM, James McMahon <jsmcmahon3@gmail.com> wrote:

Hello I= am trying to use a Publish AMQP step in my NiFi workflow. I generate a flo= w file with random text content=C2=A0using Generate Flow File, and then sen= d to a rabbitMQ Exchange named nifiAbcX using Publish AMQP. My exchange is bound to queue AbcQ by k= ey *.Abc.*, and my virtual host is nifiVhost. I have established my = exchange as a Topic type.

=C2=A0

My NiFi= instance and my RabbitMQ instance are co-located for development test purp= oses on the same server.

=C2=A0

It appe= ars that my Publish AMQP does indeed generate output which it sends down th= e "success" output flow path, but at the RabbitMQ Management Cons= ole I find that my Exchange and my Queue are Idle and show no messages. My nifi log shows this Warning mes= sage: Failure while closing target resource AMQPublisher:amqp://test@127.0.0.1= :5673/nifiVhost, EXCHANGE:nifiAbcX, ROUTING_KEY:xyz.Abc.prod=C2=A0 What is thi= s telling me? Why=C2=A0am I seeing no messages posted to my rabbitMQ queue = through the exchange? Thank you in advance.








--001a11c15902661a530543284eb4--