nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joseph Witt (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-821) Support Merging of Avro
Date Wed, 05 Aug 2015 20:11:04 GMT

    [ https://issues.apache.org/jira/browse/NIFI-821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14658804#comment-14658804
] 

Joseph Witt commented on NIFI-821:
----------------------------------

I would eliminate the possibility of issue by having that merge strategy expect an attribute
called 'avro.schema' and it will only merge items together which have a matching 'avro.schema'.
 This would be a further discriminator beyond the existing 'correlation attribute'.  You could
technically just reuse the 'correlation attribute' as well.  But something like this I think
is the right answer.  Make it explicit and eliminate this possible confusion.  One way to
set the 'avro.schema' would be some processor which characterizes the avro by perhaps extracting
the schema to an attribute and in the process also produces a hash of that schema (which thus
becomes the 'avro.schema' value).

Thanks
Joe

> Support Merging of Avro
> -----------------------
>
>                 Key: NIFI-821
>                 URL: https://issues.apache.org/jira/browse/NIFI-821
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Bryan Bende
>            Assignee: Bryan Bende
>            Priority: Minor
>             Fix For: 0.4.0
>
>         Attachments: NIFI-821.patch
>
>
> We should support the ability to merge Avro files of the same schema, similar to how
MergeContent works.
> Avro tools provides a command line tool for doing this which can be found here: 
> https://github.com/apache/avro/blob/trunk/lang/java/tools/src/main/java/org/apache/avro/tool/ConcatTool.java



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message