nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tony Kurc (JIRA)" <>
Subject [jira] [Commented] (NIFI-945) New property (wrap as array) in avro2json converter
Date Mon, 19 Oct 2015 02:45:05 GMT


Tony Kurc commented on NIFI-945:

Reviewed pull request, and made some modifications. I left out guava and had propertydescriptor
initialization in same style as other processors (which left it only there for a test annotation).
Really struggled with whitespace on this pr for some reason. Rebased it off of master. branch
"pr/945" is ready for review.

Also modified the properties to align with NIFI-967, opening the door for other container
options like [~rdblue] mentioned.

> New property (wrap as array) in avro2json converter
> ---------------------------------------------------
>                 Key: NIFI-945
>                 URL:
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>            Reporter: Joe Mészáros
>            Assignee: Tony Kurc
>            Priority: Minor
>              Labels: avro, improvement, json
> Create a new property (wrap as array) in ConvertAvroToJson, which determines how stream
of records is exposed: either as a sequence of single Objects (false), writing every Object
to a new line, or as an array of Objects. Default value is true, meaning that the Avro content
is exposed as a sequence of root-level Object entries. False value is useful, when you want
to write your records as single instances to a target component (e.g. Kafka).
> Let's assume you have an Avro content as stream of events: {noformat}({"id":"user1",
"item":"itemX", "action": "buy"}, {"id":"user2", "item":"itemY", "action": "like"}){noformat}
> If wrap as array is false, the converter will expose the records as sequence of single
JSON objects:
> {noformat}
> {"id":"user1", "item":"itemX", "action": "buy"}
> {"id":"user2", "item":"itemY", "action": "like"}
> ...
> {"id":"userN", ...}
> {noformat}
> Please bear in mind, that the final output is not a valid JSON content. You can then
forward this content e.g. to Kafka, where every record will be a single Kafka message.
> If wrap as array is true, the output looks like this:
> {noformat}
> [{"id":"user1", "item":"itemX", "action": "buy"},{"id":"user2", "item":"itemY", "action":
"like"},...,{"id":"userN", ...}]
> {noformat}
> It is useful when you want to convert your Avro content to a valid JSON array.

This message was sent by Atlassian JIRA

View raw message