nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Payne (JIRA)" <>
Subject [jira] [Commented] (NIFI-551) Improve error handling for ConvertJSONToAvro processor
Date Thu, 30 Apr 2015 23:49:06 GMT


Mark Payne commented on NIFI-551:

[~rickysaltzer]: my understanding is that in ApacheLand attaching the fix here is the preferred
way over Github.

I looked over the patch you provided. I'm concerned a bit about the way that it handles failures.
If a record cannot be converted to Avro, there is an error message sent down the 'failure'
relationship, but the record that failed to convert is actually lost. Typically, what we would
do in this situation is to route a FlowFile to 'failure' but that FlowFile would contain the
actual message that could not be converted, and then we would just log an error so that the
user sees the bulletin.

Can you explain the use case better?

> Improve error handling for ConvertJSONToAvro processor
> ------------------------------------------------------
>                 Key: NIFI-551
>                 URL:
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: Ricky Saltzer
>              Labels: patch, review
>             Fix For: 0.1.0
>         Attachments: NIFI-551.1.patch, NIFI-551.2.patch
> Currently, if the ConvertJSONToAvro processor fails to process an individual record,
a counter is incremented, but no alerts are produced. It would be better to notify the bulletin
board that we've failed to process some records for a flowfile. Further, we should stream
the records we fail to process down the failure relationship for further inspection. 

This message was sent by Atlassian JIRA

View raw message