nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <>
Subject [jira] [Commented] (NIFI-1356) Create Avro Schema by inferring CSV and JSON data
Date Tue, 26 Jan 2016 14:07:40 GMT


ASF subversion and git services commented on NIFI-1356:

Commit 68975dc2d70f46bcae3c14a0750123f012dec7ea in nifi's branch refs/heads/master from [~markap14]
[;h=68975dc ]

NIFI-1356: Fixed typo in unit test that caused it to look for Shapes_Header.csv.avro instead
of Shapes_header.csv.avro (i.e., case of h in Header)

> Create Avro Schema by inferring CSV and JSON data
> -------------------------------------------------
>                 Key: NIFI-1356
>                 URL:
>             Project: Apache NiFi
>          Issue Type: New Feature
>          Components: Extensions
>    Affects Versions: 0.4.1
>            Reporter: Jeremy Dyer
>            Assignee: Jeremy Dyer
>              Labels: features
>             Fix For: 0.5.0
>   Original Estimate: 24h
>  Remaining Estimate: 24h
> There are several situations where the ability to dynamically create an Avro schema is
desired. Kite provides the ability to dynamically infer an Avro schema from both CSV and JSON
data. Since NiFi already contains a kite bundle for converting csv and json to Avro this feature
should be an easy add. 
> I propose 2 new processors "InferAvroSchemaFromCSV" and "InferAvroSchemaFromJSON". These
processors will reside inside of the existing "nifi-kite-bundle" and extend upon the already
present third party libraries. Each processor will accept either CSV or JSON and produce as
output the Avro Schema JSON and the original data that was presented to them. The processors
will rely on kite to perform the actually inferring of the schema.

This message was sent by Atlassian JIRA

View raw message