nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pat Trainor <pat.trai...@gmail.com>
Subject Re: PutKafka won't Put! "Failed while waiting for acks from Kafka"
Date Sat, 11 Jun 2016 19:01:36 GMT
Really?! Sure! Why don't I clear everything out, and run just the generator
&amp; PutKafka, then I'll clip the logs from that so it's clean (with app
startup, etc.)...  
  
Gimme a minute to do that...  

  

Thanks a bunch for writing back!!!!  
  

[pat](http://about.me/PatTrainor)  

( ͡° ͜ʖ ͡°)  
  

On Jun 11 2016, at 1:38 pm, Joe Witt &lt;joe.witt@gmail.com&gt; wrote:  

> Pat

>

> Could you please make sure we are seeing the full stacktrace from the app
log?  If we could get a full log that would he even better.

>

> Thanks  
Joe

>

> On Jun 11, 2016 1:17 PM, "Pat Trainor"
&lt;[pat.trainor@gmail.com](mailto:pat.trainor@gmail.com)&gt; wrote:  

>

>> Andrew,

>>

>>  

>>

>> Thanks for writing.

>>

>>  

>>

>> _Nifi:_

>>

>> nifi-0.6.1

>>

>> _Kafka:_

>>

>> kafka_2.11-0.10.0.0

>>

>>  

>>

>> startup with no changes to configs:

>>

>>  

>>

>> #

>>

>> Kafka:

>>

>> bin/zookeeper-server-start.sh config/zookeeper.properties &amp;  
bin/kafka-server-start.sh config/server.properties &amp;  
#  
# create a topic  
#  
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1
--partitions 1 --topic test  

>>

>> [...]

>>

>>  

>>

>> _Nifi's zookeeper.properties:_

>>

>>  

>>

>>  

>>

>> clientPort=2181  
initLimit=10  
autopurge.purgeInterval=24  
syncLimit=5  
tickTime=2000  
dataDir=./state/zookeeper  
autopurge.snapRetainCount=30  
  
#  
# server.1=nifi-node1-hostname:2888:3888  
# server.2=nifi-node2-hostname:2888:3888  
# server.3=nifi-node3-hostname:2888:3888  
#  
**server.1=  
**
>>

>>  

>>

>> ^^^^^^

>>

>> Even though I have:

>>

>>  

>>

>> _nifi.state.management.embedded.zookeeper.start=**false**  
_

>>

>>  

>>

>> I filled in the above setting... changing it to:

>>

>>  

>>

>> **server.1=localhost:2888:3888  
**
>>

>>  

>>

>> ...which made no difference, of course...

>>

>>  

>>

>> Any thoughts?

>>

>>  

>>

>> Thanks!  
  

>>

>> [pat](http://about.me/PatTrainor)  

>>

>> ( ͡° ͜ʖ ͡°)  
  

>>

>> On Jun 11 2016, at 12:17 pm, Andrew Grande
&lt;[aperepel@gmail.com](mailto:aperepel@gmail.com)&gt; wrote:  

>>

>>> Pat, which NiFi version is that? Which Kafka broker version on the other
end?

>>>

>>>  

>>>

>>> Andrew

>>>

>>>  

>>>

>>> On Sat, Jun 11, 2016 at 12:15 PM Pat Trainor
&lt;[pat.trainor@gmail.com](mailto:pat.trainor@gmail.com)&gt; wrote:  

>>>

>>>> Users,

>>>>

>>>>  

>>>>

>>>> Has anyone come across PutKafka simply not wanting to send to Kafka?

>>>>

>>>>  

>>>>

>>>> In this troubleshooting, I have replicated symptoms in the simplest way I
can-with a GenerateFlowFile and a PutKafka:

>>>>

>>>>  

>>>>

>>>> ![](https://community.hortonworks.com/storage/temp/4972-screenshot-
from-2016-06-11-11-34-25.png)

>>>>

>>>> With a known good Kafka running (other /producers using the same
Topic)...

>>>>

>>>> ...yields:

>>>>

>>>> Auto refresh started

>>>>

>>>> 11:17:19 EDT WARNING 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76
634c-6354-4c3c-b977-86f73d64efbc] Processor Administratively Yielded for 1 sec
due to processing failure

>>>>

>>>> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b7663
4c-6354-4c3c-b977-86f73d64efbc] Failed while waiting for acks from Kafka

>>>>

>>>> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b7663
4c-6354-4c3c-b977-86f73d64efbc]
PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process due to
java.lang.NullPointerException; rolling back session:
java.lang.NullPointerException

>>>>

>>>> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbc

>>>>

>>>> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc]
PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process session
due to java.lang.NullPointerException: java.lang.NullPointerException

>>>>

>>>>  

>>>>

>>>> Everything is super-stock, out of the box, and not customized. In fact, I
tested Kafka with the Apache test consumer/producer pair, and am trying to
send to the same topic, same consumer, still running, after it worked for the
test producer.

>>>>

>>>>  

>>>>

>>>> Funny thing is, I see this error in forums/emails lately, but nobody
nails it down as to why it is occurring...

>>>>

>>>>  

>>>>

>>>> ==&gt; nifi-app.log &lt;==  
2016-06-11 12:04:24,172 WARN [kafka-producer-network-thread |
SecondPutKafka2k] org.apache.kafka.common.network.Selector Error in I/O with
localhost/[127.0.0.1](http://127.0.0.1)  
java.io.EOFException: null  
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java
:62) ~[kafka-clients-0.8.2.2.jar:na]  
at org.apache.kafka.common.network.Selector.poll(Selector.java:248) ~[kafka-
clients-0.8.2.2.jar:na]  
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) [kafka-
clients-0.8.2.2.jar:na]  
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
[kafka-clients-0.8.2.2.jar:na]  
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
[kafka-clients-0.8.2.2.jar:na]  
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]  

>>>>

>>>>  

>>>>

>>>> attached has screenshots...

>>>>

>>>>  

>>>>

>>>> I've slowed down the 'schedules', messes with Message Delimiter, and what
seems like all the obvious things... This makes me think it is obvious, so on
day 2, I'm reaching out to see if any of you have seen/fixed this...

>>>>

>>>>  

>>>>

>>>> Thanks!  
  

>>>>

>>>> [pat](http://about.me/PatTrainor)  

>>>>

>>>> ( ͡° ͜ʖ ͡°)  
  

Mime
View raw message