nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-4004) Refactor RecordReaderFactory and SchemaAccessStrategy to be used without incoming FlowFile
Date Fri, 08 Sep 2017 16:40:00 GMT

    [ https://issues.apache.org/jira/browse/NIFI-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158892#comment-16158892
] 

ASF GitHub Bot commented on NIFI-4004:
--------------------------------------

Github user markap14 commented on the issue:

    https://github.com/apache/nifi/pull/1877
  
    @ijokarumawak thanks for the updated PR! All looks good at this point. +1 Have merged
to master.


> Refactor RecordReaderFactory and SchemaAccessStrategy to be used without incoming FlowFile
> ------------------------------------------------------------------------------------------
>
>                 Key: NIFI-4004
>                 URL: https://issues.apache.org/jira/browse/NIFI-4004
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>    Affects Versions: 1.2.0
>            Reporter: Koji Kawamura
>            Assignee: Koji Kawamura
>             Fix For: 1.4.0
>
>
> Current RecordReaderFactory and SchemaAccessStrategy implementation assumes there's always
an incoming FlowFile available, and use it to resolve Record Schema.
> That is fine for components those convert or update incoming FlowFiles, however there
are other components those does not have any incoming FlowFiles, for example, ConsumeKafkaRecord_0_10.
Typically, ones fetches data from external system do not have incoming FlowFile. And current
API doesn't fit well with these as it requires a FlowFile.
> In fact, [ConsumeKafkaRecord creates a temporal FlowFile|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-10-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumerLease.java#L426]
only to get RecordSchema. This should be avoided as we expect more components start using
Record reader mechanism.
> This JIRA proposes refactoring current API to allow accessing RecordReaders without needing
an incoming FlowFile.
> Additionally, since there's Schema Access Strategy that requires incoming FlowFile containing
attribute values to access schema registry, it'd be useful if we could tell user when such
RecordReader is specified that it can't be used.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message