nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Blue (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-919) Support Splitting Avro Files
Date Fri, 04 Sep 2015 18:22:45 GMT

    [ https://issues.apache.org/jira/browse/NIFI-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14731191#comment-14731191
] 

Ryan Blue commented on NIFI-919:
--------------------------------

I think we should go with record strategy to start off with. The block strategy doesn't sound
very useful to me since users don't typically know anything about Avro blocks. They are almost
always an internal thing that users don't see. Users might want to break the file into chunks
based on a target file size, which I think could be a good use case to support and makes AVRO-1726
a bit easier to implement as well.

> Support Splitting Avro Files
> ----------------------------
>
>                 Key: NIFI-919
>                 URL: https://issues.apache.org/jira/browse/NIFI-919
>             Project: Apache NiFi
>          Issue Type: New Feature
>            Reporter: Bryan Bende
>            Assignee: Bryan Bende
>            Priority: Minor
>             Fix For: 0.4.0
>
>
> Provide a processor that splits an Avro file into multiple smaller files. Would be nice
to have a configurable batch size so a user could produce single record files and also multi-record
files of smaller size than the original. Also consider making the output format configurable,
data file vs bare record.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message