nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-3431) Support batch update in Notify processor
Date Thu, 02 Feb 2017 06:26:51 GMT

    [ https://issues.apache.org/jira/browse/NIFI-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849501#comment-15849501
] 

ASF GitHub Bot commented on NIFI-3431:
--------------------------------------

GitHub user ijokarumawak opened a pull request:

    https://github.com/apache/nifi/pull/1466

    NIFI-3431: Support batch update in Notify processor

    - Added Signal Counter Delta property
    - Added Signal Buffer Count property
    - Changed IOException handling from routing it to failure to throw
      RuntimeException, so that NiFi framework can yield the processor for a while and try
again
    
    Thank you for submitting a contribution to Apache NiFi.
    
    In order to streamline the review of the contribution we ask you
    to ensure the following steps have been taken:
    
    ### For all changes:
    - [x] Is there a JIRA ticket associated with this PR? Is it referenced 
         in the commit message?
    
    - [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying
to resolve? Pay particular attention to the hyphen "-" character.
    
    - [x] Has your PR been rebased against the latest commit within the target branch (typically
master)?
    
    - [x] Is your initial contribution a single, squashed commit?
    
    ### For code changes:
    - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check
clean install at the root nifi folder?
    - [x] Have you written or updated unit tests to verify your changes?
    - [ ] If adding new dependencies to the code, are these dependencies licensed in a way
that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?

    - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file
under nifi-assembly?
    - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file
found under nifi-assembly?
    - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic
access) for each of the new properties?
    
    ### For documentation related changes:
    - [x] Have you ensured that format looks appropriate for the output in which it is rendered?
    
    ### Note:
    Please ensure that once the PR is submitted, you check travis-ci for build issues and
submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ijokarumawak/nifi nifi-3431

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/nifi/pull/1466.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1466
    
----
commit c0e6190bb95d8ee701801942ec07cc2c2ee80923
Author: Koji Kawamura <ijokarumawak@apache.org>
Date:   2017-02-02T06:18:01Z

    NIFI-3431: Support batch update in Notify processor
    
    - Added Signal Counter Delta property
    - Added Signal Buffer Count property
    - Changed IOException handling from routing it to failure to throw
      RuntimeException, so that NiFi framework can yield the processor for a while and try
again

----


> Support batch update in Notify processor
> ----------------------------------------
>
>                 Key: NIFI-3431
>                 URL: https://issues.apache.org/jira/browse/NIFI-3431
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>    Affects Versions: 1.2.0
>            Reporter: Koji Kawamura
>            Assignee: Koji Kawamura
>
> NIFI-3216 added ability to wait for N signals. It supports waiting for N fragments split
by SplitXXXX processors. However, since Notify processor has to increase count one by one
by calling expensive replace cache operation over network, it doesn't provide a practical
performance when user configured a flow looks like below as N glow:
> {code}
> Split
>  --> (original) -> Wait for N
>  --> (split) -> Do something -> Notify 1
> {code}
> This JIRA improves Notify processor by:
> - Add "Signal Buffer Count" to specify max number of flow files that can be buffered
and update cache at once
> - Add "Signal Counter Delta" to specify delta grater than 1, EL supported
> "Signal Buffer Count" would be useful in a flow like this:
> {code}
> Split
>  --> (original) -> Wait for N
>  --> (split) -> Filter or something -> Notify M
> {code}
> Buffer incoming M flow files and perform cache replace operation once.
> So does "Signal Counter Delta":
> {code}
> Split
>  --> (original) -> Wait for N
>  --> (split) -> Filter or something -> Merge M -> PutXXX -> Notify M
> {code}
> Specify 'M' via Attribute Expression Language.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message