nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject Re: PutKakfa can send a file to its success output although the Kafka put fails
Date Thu, 25 Aug 2016 20:50:44 GMT
Def seems like a good candidate for a bug report.

Thanks
Joe

On Thu, Aug 25, 2016 at 4:45 PM, McDermott, Chris Kevin (MSDU -
STaTS/StorefrontRemote) <chris.mcdermott@hpe.com> wrote:

> I just wanted to sanity check this before opening a JIRA.
>
>
>
> I have PutKafka processor where I’ve set the Max Record Size property to
> 1MB.
>
>
>
> If a transfer larger file to it, I see a bulletin and the following
> message in the log file, but the flowfile is transferred to the success
> output.
>
>
>
> Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The
> message is 5800160 bytes when serialized which is larger than the maximum
> request size you have configured with the max.request.si\
>
> ze configuration.
>
> 2016-08-25 20:38:22,041 ERROR [Timer-Driven Process Thread-7]
> o.apache.nifi.processors.kafka.PutKafka PutKafka[id=3c396e30-7294-467e-b5d7-a3d44c0a04f4]
> Failed while waiting for acks from Kafka
>
> 2016-08-25 20:38:22,042 ERROR [Timer-Driven Process Thread-7]
> o.apache.nifi.processors.kafka.PutKafka
>
> java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.RecordTooLargeException:
> The message is 5800160 bytes when serialized which is larger than the
> maximum request size you have conf\
>
> igured with the max.request.size configuration.
>
>         at org.apache.kafka.clients.producer.KafkaProducer$
> FutureFailure.<init>(KafkaProducer.java:437) ~[kafka-clients-0.8.2.2.jar:
> na]
>
>         at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:352)
> ~[kafka-clients-0.8.2.2.jar:na]
>
>         at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:248)
> ~[kafka-clients-0.8.2.2.jar:na]
>
>         at org.apache.nifi.processors.kafka.KafkaPublisher.publish(KafkaPublisher.java:137)
> ~[nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
>
>         at org.apache.nifi.processors.kafka.PutKafka$1.process(PutKafka.java:315)
> [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
>
>         at org.apache.nifi.controller.repository.
> StandardProcessSession.read(StandardProcessSession.java:1851)
> [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
>
>         at org.apache.nifi.controller.repository.
> StandardProcessSession.read(StandardProcessSession.java:1822)
> [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
>
>         at org.apache.nifi.processors.kafka.PutKafka.
> doRendezvousWithKafka(PutKafka.java:311) [nifi-kafka-processors-oculus-
> 0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
>
>         at org.apache.nifi.processors.kafka.PutKafka.
> rendezvousWithKafka(PutKafka.java:287) [nifi-kafka-processors-oculus-
> 0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
>
>         at org.apache.nifi.processors.kafka.AbstractKafkaProcessor.
> onTrigger(AbstractKafkaProcessor.java:76) [nifi-kafka-processors-oculus-
> 0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
>
>         at org.apache.nifi.controller.StandardProcessorNode.onTrigger(
> StandardProcessorNode.java:1054) [nifi-framework-core-oculus-0.
> 7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
>
>         at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.
> call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-oculus-0.
> 7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
>
>         at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.
> call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-oculus-0.
> 7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
>
>         at org.apache.nifi.controller.scheduling.
> TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
> [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
>
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
>
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_45]
>
>
>
> I have not yet tried the ProduceKafka processor.
>
>
>
> Anyone see a reason I should not open a JIRA?
>
>
>
>
>
> Chris McDermott
>
>
>
> Remote Business Analytics
>
> STaTS/StoreFront Remote
>
> HPE Storage
>
> Hewlett Packard Enterprise
>
> Mobile: +1 978-697-5315
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message