Mailing-List: contact issues-help@nifi.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@nifi.apache.org
Date: Thu, 2 Feb 2017 11:50:51 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@nifi.apache.org
Message-ID: <JIRA.13011070.1476112411000.16349.1486036251517@Atlassian.JIRA>
In-Reply-To: <JIRA.13011070.1476112411000@Atlassian.JIRA>
References: <JIRA.13011070.1476112411000@Atlassian.JIRA> <JIRA.13011070.1476112411285@jira-lw-us.apache.org>
Subject: [jira] [Commented] (NIFI-2881) Allow Database Fetch processor(s) to
 accept incoming flow files and use Expression Language
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 02 Feb 2017 11:50:59 -0000


    [ https://issues.apache.org/jira/browse/NIFI-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849827#comment-15849827 ] 

ASF GitHub Bot commented on NIFI-2881:
--------------------------------------

Github user ijokarumawak commented on a diff in the pull request:

    https://github.com/apache/nifi/pull/1407#discussion_r99103639
  
    --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AbstractDatabaseFetchProcessor.java ---
    @@ -117,9 +124,11 @@
                         + "can be used to retrieve only those rows that have been added/updated since the last retrieval. Note that some "
                         + "JDBC types such as bit/boolean are not conducive to maintaining maximum value, so columns of these "
                         + "types should not be listed in this property, and will result in error(s) during processing. If no columns "
    -                    + "are provided, all rows from the table will be considered, which could have a performance impact.")
    +                    + "are provided, all rows from the table will be considered, which could have a performance impact.\nNOTE: If Expression Language is "
    +                    + "present for this property and it refers to flow file attribute(s), then the Table Name property must also contain Expression Language.")
    --- End diff --
    
    Is this going to be updated?


> Allow Database Fetch processor(s) to accept incoming flow files and use Expression Language
> -------------------------------------------------------------------------------------------
>
>                 Key: NIFI-2881
>                 URL: https://issues.apache.org/jira/browse/NIFI-2881
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: Matt Burgess
>            Assignee: Matt Burgess
>
> The QueryDatabaseTable and GenerateTableFetch processors do not allow Expression Language to be used in the properties, mainly because they also do not allow incoming connections. This means if the user desires to fetch from multiple tables, they currently need one instance of the processor for each table, and those table names must be hard-coded.
> To support the same capabilities for multiple tables and more flexible configuration via Expression Language, these processors should have properties that accept Expression Language, and GenerateTableFetch should accept (optional) incoming connections.
> Conversation about the behavior of the processors is welcomed and encouraged. For example, if an incoming flow file is available, do we also still run the incremental fetch logic for tables that aren't specified by this flow file, or do we just do incremental fetching when the processor is scheduled but there is no incoming flow file. The latter implies a denial-of-service could take place, by flooding the processor with flow files and not letting it do its original job of querying the table, keeping track of maximum values, etc.
> This is likely a breaking change to the processors because of how state management is implemented. Currently since the table name is hard coded, only the column name comprises the key in the state. This would have to be extended to have a compound key that represents table name, max-value column name, etc.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)