nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Hunyadi (Jira)" <j...@apache.org>
Subject [jira] [Updated] (MINIFICPP-1251) Implement RetryFlowFile Processor
Date Tue, 16 Jun 2020 13:22:00 GMT

     [ https://issues.apache.org/jira/browse/MINIFICPP-1251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Adam Hunyadi updated MINIFICPP-1251:
------------------------------------
    Attachment:     (was: RetryFlowFileAcceptanceTable.numbers)

> Implement RetryFlowFile Processor
> ---------------------------------
>
>                 Key: MINIFICPP-1251
>                 URL: https://issues.apache.org/jira/browse/MINIFICPP-1251
>             Project: Apache NiFi MiNiFi C++
>          Issue Type: New Feature
>            Reporter: Adam Hunyadi
>            Assignee: Adam Hunyadi
>            Priority: Minor
>         Attachments: RetryFlowFileAcceptanceTable.numbers, RetryFlowFileAcceptanceTable.pdf
>
>
> *Acceptance criteria**:*
> Planned flow to use:
> *GetFile(s)* => *PutFile* (with *Conflict Resolution Strategy* set to *Fail*) =>
*RetryFlowFile*
> Using this conflict resolution strategy on PutFile to fail means that we can make it
fail by making it try writing a file that already exists.
>  
> See attached table for a table that is easier to read than this one:
> |{color:#000000}*Expected Retry Property Name*{color}|{color:#000000}*Expected Retry
Property Value*{color}|{color:#000000}*Expected Outbound Relationship*{color}|{color:#000000}*Expect
Penalty on Flowfile*{color}|{color:#000000}*Retry Attribute Value on FlowFile*{color}|{color:#000000}*Retry
Attribute Value Before Processing*{color}|{color:#000000}*Maximum Retries*{color}|{color:#000000}*Penalize
Retries*{color}|{color:#000000}*Fail on Non-numerical Overwrite*{color}|{color:#000000}*Reuse
Mode*{color}|{color:#000000}*Processor UUID matches FlowFile UUID*{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*1*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}n/a{color}|
> |{color:#000000}*flowfile.retryCount*{color}|{color:#000000}*1*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}flowfile.retryCount{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}n/a{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*2*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}1{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}(not
set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*(property cleared)*{color}|{color:#000000}*(property cleared)*{color}|{color:#000000}*retries_exceeded*{color}|{color:#000000}*FALSE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}3{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}(not
set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*(property cleared)*{color}|{color:#000000}*(property cleared)*{color}|{color:#000000}*retries_exceeded*{color}|{color:#000000}*FALSE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}4{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}(not
set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*6*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}5{color}|{color:#000000}6{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*1*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}2{color}|{color:#000000}(not
set){color}|{color:#000000}TRUE{color}|{color:#000000}(not set){color}|{color:#000000}(not
set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*(property cleared)*{color}|{color:#000000}*(property cleared)*{color}|{color:#000000}*retries_exceeded*{color}|{color:#000000}*FALSE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}3{color}|{color:#000000}(not
set){color}|{color:#000000}TRUE{color}|{color:#000000}(not set){color}|{color:#000000}(not
set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*1*{color}|{color:#000000}*retry*{color}|{color:#000000}*FALSE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}2{color}|{color:#000000}(not
set){color}|{color:#000000}FALSE{color}|{color:#000000}(not set){color}|{color:#000000}(not
set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*(property cleared)*{color}|{color:#000000}*(property cleared)*{color}|{color:#000000}*retries_exceeded*{color}|{color:#000000}*FALSE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}3{color}|{color:#000000}(not
set){color}|{color:#000000}FALSE{color}|{color:#000000}(not set){color}|{color:#000000}(not
set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*2*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}1{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}TRUE{color}|{color:#000000}(not
set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*2*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}1{color}|{color:#000000}6{color}|{color:#000000}(not
set){color}|{color:#000000}FALSE{color}|{color:#000000}(not set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*1*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}“incorrect”{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}(not
set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*(property cleared)*{color}|{color:#000000}*failure*{color}|{color:#000000}*FALSE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}“incorrect”{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}TRUE{color}|{color:#000000}(not
set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*1*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}“incorrect”{color}|{color:#000000}6{color}|{color:#000000}(not
set){color}|{color:#000000}FALSE{color}|{color:#000000}(not set){color}|{color:#000000}TRUE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*2*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}flowfile.retries{color}|{color:#000000}1{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}“Fail
On Reuse”{color}|{color:#000000}TRUE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*2*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}{color:#000000}flowfile.retries{color}{color}|{color:#000000}1{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}“Warn
on Reuse”{color}|{color:#000000}TRUE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*2*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}{color:#000000}flowfile.retries{color}{color}|{color:#000000}1{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}“Reset
Reuse”{color}|{color:#000000}TRUE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*1*{color}|{color:#000000}*failure*{color}|{color:#000000}*FALSE*{color}|{color:#000000}{color:#000000}flowfile.retries{color}{color}|{color:#000000}1{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}(not
set){color}|{color:#000000}FALSE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*1*{color}|{color:#000000}*failure*{color}|{color:#000000}*FALSE*{color}|{color:#000000}flowfile.{color:#000000}retries{color}{color}|{color:#000000}1{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}“Fail
On Reuse”{color}|{color:#000000}FALSE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*1*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}flowfile.{color:#000000}retries{color}{color}|{color:#000000}1{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}“Warn
on Reuse”{color}|{color:#000000}FALSE{color}|
> |{color:#000000}*flowfile.retries*{color}|{color:#000000}*1*{color}|{color:#000000}*retry*{color}|{color:#000000}*TRUE*{color}|{color:#000000}flowfile.{color:#000000}retries{color}{color}|{color:#000000}1{color}|{color:#000000}(not
set){color}|{color:#000000}(not set){color}|{color:#000000}(not set){color}|{color:#000000}“Reset
Reuse”{color}|{color:#000000}FALSE{color}|
>  
> Other:
> GIVEN the reuse mode is set to *{color:#000000}“Warn on Reuse”{color}*
>  WHEN the flowfile already has a retries attribute generated by a different processor
>  THEN a log line should mention this on *warning* level
> *Background:*
> NiFi already has a RetryFlowFile processor, as documented here:
> [[Apache documentation for RetryFlowFile]|http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.RetryFlowFile/index.html]
>  
> *Proposal:*
> As this is an important logic for creating flows, we should port this functionality to
MiNiFi as well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message