nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-3431) Support batch update in Notify processor
Date Thu, 02 Feb 2017 09:01:03 GMT

    [ https://issues.apache.org/jira/browse/NIFI-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849676#comment-15849676
] 

ASF GitHub Bot commented on NIFI-3431:
--------------------------------------

Github user ijokarumawak commented on the issue:

    https://github.com/apache/nifi/pull/1466
  
    Dear reviewer, I wrote a blog post on how to use Wait/Notify processors which can take
benefit from this PR. I hope this helps:
    http://ijokarumawak.github.io/nifi/2017/02/02/nifi-notify-batch/


> Support batch update in Notify processor
> ----------------------------------------
>
>                 Key: NIFI-3431
>                 URL: https://issues.apache.org/jira/browse/NIFI-3431
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>    Affects Versions: 1.2.0
>            Reporter: Koji Kawamura
>            Assignee: Koji Kawamura
>
> NIFI-3216 added ability to wait for N signals. It supports waiting for N fragments split
by SplitXXXX processors. However, since Notify processor has to increase count one by one
by calling expensive replace cache operation over network, it doesn't provide a practical
performance when user configured a flow looks like below as N glow:
> {code}
> Split
>  --> (original) -> Wait for N
>  --> (split) -> Do something -> Notify 1
> {code}
> This JIRA improves Notify processor by:
> - Add "Signal Buffer Count" to specify max number of flow files that can be buffered
and update cache at once
> - Add "Signal Counter Delta" to specify delta grater than 1, EL supported
> "Signal Buffer Count" would be useful in a flow like this:
> {code}
> Split
>  --> (original) -> Wait for N
>  --> (split) -> Filter or something -> Notify M
> {code}
> Buffer incoming M flow files and perform cache replace operation once.
> So does "Signal Counter Delta":
> {code}
> Split
>  --> (original) -> Wait for N
>  --> (split) -> Filter or something -> Merge M -> PutXXX -> Notify M
> {code}
> Specify 'M' via Attribute Expression Language.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message