nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <chris.mcderm...@hpe.com>
Subject Re: PutKakfa can send a file to its success output although the Kafka put fails
Date Fri, 26 Aug 2016 17:16:57 GMT
https://issues.apache.org/jira/browse/NIFI-2680

Chris McDermott
 
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
 


On 8/25/16, 4:50 PM, "Joe Witt" <joe.witt@gmail.com> wrote:

    Def seems like a good candidate for a bug report.
    
    Thanks
    Joe
    
    On Thu, Aug 25, 2016 at 4:45 PM, McDermott, Chris Kevin (MSDU -
    STaTS/StorefrontRemote) <chris.mcdermott@hpe.com> wrote:
    
    > I just wanted to sanity check this before opening a JIRA.
    >
    >
    >
    > I have PutKafka processor where I’ve set the Max Record Size property to
    > 1MB.
    >
    >
    >
    > If a transfer larger file to it, I see a bulletin and the following
    > message in the log file, but the flowfile is transferred to the success
    > output.
    >
    >
    >
    > Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The
    > message is 5800160 bytes when serialized which is larger than the maximum
    > request size you have configured with the max.request.si\
    >
    > ze configuration.
    >
    > 2016-08-25 20:38:22,041 ERROR [Timer-Driven Process Thread-7]
    > o.apache.nifi.processors.kafka.PutKafka PutKafka[id=3c396e30-7294-467e-b5d7-a3d44c0a04f4]
    > Failed while waiting for acks from Kafka
    >
    > 2016-08-25 20:38:22,042 ERROR [Timer-Driven Process Thread-7]
    > o.apache.nifi.processors.kafka.PutKafka
    >
    > java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.RecordTooLargeException:
    > The message is 5800160 bytes when serialized which is larger than the
    > maximum request size you have conf\
    >
    > igured with the max.request.size configuration.
    >
    >         at org.apache.kafka.clients.producer.KafkaProducer$
    > FutureFailure.<init>(KafkaProducer.java:437) ~[kafka-clients-0.8.2.2.jar:
    > na]
    >
    >         at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:352)
    > ~[kafka-clients-0.8.2.2.jar:na]
    >
    >         at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:248)
    > ~[kafka-clients-0.8.2.2.jar:na]
    >
    >         at org.apache.nifi.processors.kafka.KafkaPublisher.publish(KafkaPublisher.java:137)
    > ~[nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
    >
    >         at org.apache.nifi.processors.kafka.PutKafka$1.process(PutKafka.java:315)
    > [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
    >
    >         at org.apache.nifi.controller.repository.
    > StandardProcessSession.read(StandardProcessSession.java:1851)
    > [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
    >
    >         at org.apache.nifi.controller.repository.
    > StandardProcessSession.read(StandardProcessSession.java:1822)
    > [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
    >
    >         at org.apache.nifi.processors.kafka.PutKafka.
    > doRendezvousWithKafka(PutKafka.java:311) [nifi-kafka-processors-oculus-
    > 0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
    >
    >         at org.apache.nifi.processors.kafka.PutKafka.
    > rendezvousWithKafka(PutKafka.java:287) [nifi-kafka-processors-oculus-
    > 0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
    >
    >         at org.apache.nifi.processors.kafka.AbstractKafkaProcessor.
    > onTrigger(AbstractKafkaProcessor.java:76) [nifi-kafka-processors-oculus-
    > 0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
    >
    >         at org.apache.nifi.controller.StandardProcessorNode.onTrigger(
    > StandardProcessorNode.java:1054) [nifi-framework-core-oculus-0.
    > 7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
    >
    >         at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.
    > call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-oculus-0.
    > 7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
    >
    >         at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.
    > call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-oculus-0.
    > 7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
    >
    >         at org.apache.nifi.controller.scheduling.
    > TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
    > [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
    >
    >         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    > [na:1.8.0_45]
    >
    >         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    > [na:1.8.0_45]
    >
    >
    >
    > I have not yet tried the ProduceKafka processor.
    >
    >
    >
    > Anyone see a reason I should not open a JIRA?
    >
    >
    >
    >
    >
    > Chris McDermott
    >
    >
    >
    > Remote Business Analytics
    >
    > STaTS/StoreFront Remote
    >
    > HPE Storage
    >
    > Hewlett Packard Enterprise
    >
    > Mobile: +1 978-697-5315
    >
    >
    >
    >
    

Mime
View raw message