nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-627) ControlRate processor does not accurately control the rate
Date Sun, 13 Mar 2016 17:29:33 GMT

    [ https://issues.apache.org/jira/browse/NIFI-627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15192426#comment-15192426
] 

ASF subversion and git services commented on NIFI-627:
------------------------------------------------------

Commit 0cb4047adde5f82a1554fa13431909c5e5eb4f25 in nifi's branch refs/heads/master from [~boardm26]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=0cb4047 ]

NIFI-627 removed flowfile penalization which could skew behavior when processor's Time Duration
was less than Penalty Duration, improved over throttle penalization NIFI-990 corrected failure
path NIFI-1329 refactored using FlowFileFilter to avoid repeatedly returning flowfiles to
the input queue, producing misleading stats and excessive Tasks/Time used

Signed-off-by: joewitt <joewitt@apache.org>


> ControlRate processor does not accurately control the rate
> ----------------------------------------------------------
>
>                 Key: NIFI-627
>                 URL: https://issues.apache.org/jira/browse/NIFI-627
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 0.1.0
>            Reporter: Michael Moser
>            Assignee: Michael Moser
>            Priority: Minor
>             Fix For: 0.6.0
>
>
> Set a ControlRate processor to something like 5 KB per 1 sec.  Generate flow files that
are about 300 KB in size and feed a bunch to this processor.  This should allow about 5 files
through per 5 minutes.  But it allows a lot more data through than it should.  The difference
seems to get worse with really low Time Duration values.  And people tend to think in number
of bytes per second so the temptation to set Time Duration to 1 sec is great.
> Also, if ControlRate has multiple input queues, it seems to output even more data than
it should.
> This seems to be caused by the code at the beginning of ControlRate onTrigger().  Under
some conditions when the number of files that are allowed through per Time Duration is less
than 1, the Throttle is being removed from the throttleMap while it actually still should
be in use.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message