nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Mészáros (JIRA) <j...@apache.org>
Subject [jira] [Updated] (NIFI-945) New property (wrap as array) in avro2json converter
Date Thu, 10 Sep 2015 11:59:46 GMT

     [ https://issues.apache.org/jira/browse/NIFI-945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Joe Mészáros updated NIFI-945:
------------------------------
    Description: 
Create a new property (wrap as array) in ConvertAvroToJson, which determines how stream of
records is exposed: either as a sequence of single Objects (false), writing every Object to
a new line, or as an array of Objects. Default value is true, meaning that the Avro content
is exposed as a sequence of root-level Object entries. False value is useful, when you want
to write your records as single instances to a target component (e.g. Kafka).

Let's assume you have an Avro content as stream of events: {noformat}({"id":"user1", "item":"itemX",
"action": "buy"}, {"id":"user2", "item":"itemY", "action": "like"}){noformat}

If wrap as array is false, the converter will expose the records as sequence of single JSON
objects:
{noformat}
{"id":"user1", "item":"itemX", "action": "buy"}
{"id":"user2", "item":"itemY", "action": "like"}
...
{"id":"userN", ...}
{noformat}

Please bear in mind, that the final output is not a valid JSON content. You can then forward
this content e.g. to Kafka, where every record will be a single Kafka message.

If wrap as array is true, the output looks like this:
{noformat}
[{"id":"user1", "item":"itemX", "action": "buy"},{"id":"user2", "item":"itemY", "action":
"like"},...,{"id":"userN", ...}]
{noformat}
It is useful when you want to convert your Avro content to a valid JSON array.

  was:
Create a new property (wrap as array) in ConvertAvroToJson, which determines how stream of
records is exposed: either as a sequence of single Objects (false), writing every Object to
a new line, or as an array of Objects. Default value is true, meaning that the Avro content
is exposed as a sequence of root-level Object entries. False value is useful, when you want
to write your records as single instances to a target component (e.g. Kafka).

Let's assume you have an Avro content as stream of events {noformat}({"id":"user1", "item":"itemX",
"action": "buy"}, {"id":"user2", "item":"itemY", "action": "like"}){noformat}. If wrap as
array is false, the converter will expose the records as sequence of single JSON objects:

{"id":"user1", "item":"itemX", "action": "buy"}
{"id":"user2", "item":"itemY", "action": "like"}
...
{"id":"userN", ...}

Please bear in mind, that the final output is not a valid JSON content. You can then forward
this content e.g. to Kafka, where every record will be a single Kafka message.

If wrap as array is true, the output looks like this:
[{"id":"user1", "item":"itemX", "action": "buy"},{"id":"user2", "item":"itemY", "action":
"like"},...,{"id":"userN", ...}]

It is useful when you want to convert your Avro content to a valid JSON array.


> New property (wrap as array) in avro2json converter
> ---------------------------------------------------
>
>                 Key: NIFI-945
>                 URL: https://issues.apache.org/jira/browse/NIFI-945
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>            Reporter: Joe Mészáros
>            Priority: Minor
>              Labels: avro, improvement, json
>
> Create a new property (wrap as array) in ConvertAvroToJson, which determines how stream
of records is exposed: either as a sequence of single Objects (false), writing every Object
to a new line, or as an array of Objects. Default value is true, meaning that the Avro content
is exposed as a sequence of root-level Object entries. False value is useful, when you want
to write your records as single instances to a target component (e.g. Kafka).
> Let's assume you have an Avro content as stream of events: {noformat}({"id":"user1",
"item":"itemX", "action": "buy"}, {"id":"user2", "item":"itemY", "action": "like"}){noformat}
> If wrap as array is false, the converter will expose the records as sequence of single
JSON objects:
> {noformat}
> {"id":"user1", "item":"itemX", "action": "buy"}
> {"id":"user2", "item":"itemY", "action": "like"}
> ...
> {"id":"userN", ...}
> {noformat}
> Please bear in mind, that the final output is not a valid JSON content. You can then
forward this content e.g. to Kafka, where every record will be a single Kafka message.
> If wrap as array is true, the output looks like this:
> {noformat}
> [{"id":"user1", "item":"itemX", "action": "buy"},{"id":"user2", "item":"itemY", "action":
"like"},...,{"id":"userN", ...}]
> {noformat}
> It is useful when you want to convert your Avro content to a valid JSON array.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message