Mailing-List: contact commits-help@nifi.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@nifi.apache.org
Date: Tue, 17 May 2016 23:09:13 +0000 (UTC)
From: "ASF subversion and git services (JIRA)" <jira@apache.org>
To: commits@nifi.apache.org
Message-ID: <JIRA.12963487.1461851356000.217812.1463526553794@Atlassian.JIRA>
In-Reply-To: <JIRA.12963487.1461851356000@Atlassian.JIRA>
References: <JIRA.12963487.1461851356000@Atlassian.JIRA> <JIRA.12963487.1461851356138@arcas>
Subject: [jira] [Commented] (NIFI-1822) Allow concurrent execution of
 ExecuteScript
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 17 May 2016 23:09:15 -0000


    [ https://issues.apache.org/jira/browse/NIFI-1822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287839#comment-15287839 ] 

ASF subversion and git services commented on NIFI-1822:
-------------------------------------------------------

Commit d0c8a5d677bbea6c5e8e051df0ea49f3cf7c6a0a in nifi's branch refs/heads/0.x from [~mattyb149]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d0c8a5d ]

NIFI-1822 Enabled pooled ExecuteScript engines to facilitate concurrent processing.

Removed unused variable in unit test. (+10 squashed commits)
Squashed commits:
[7c5acc1] NIFI-1822 Removed trailing whitespace to conform with checkstyle rules.
[cb108cd] NIFI-1822 Added ASF License to unit test.
[9264428] NIFI-1822 Removed debugging log statements for script engine queue size.
Added unit tests demonstrating pooled execution timing and thread usage.
[bdbc4ba] NIFI-1822 Renamed reference to MockProcessorContext#setNumThreads to setMaxConcurrentTasks after refactor.
[12bbe82] NIFI-1822 Moved unit test to correct directory.
Added test script resource which generates flowfile and updates attribute with current thread.
Added tests for single run, serial run, and pooled run (not complete).
[4c174c8] NIFI-1822 Added debugging messages to script execution.
[8ab0ce5] NIFI-1822 Added for loop to instantiate multiple script engines in queue.
[8c5ba51] NIFI-1822 Added variable max concurrent task field in MockProcessorContext because it was previously hardcoded to 1. Changed setNumThreads to setMaxConcurrentTasks to maintain naming convention.
[fd9120c] NIFI-1822 Added unit test skeleton for pooled script processor execution.
[23e4f68] NIFI-1822: Allow concurrent execution in ExecuteScript

This closes #443.

Signed-off-by: Andy LoPresto <alopresto@apache.org>


> Allow concurrent execution of ExecuteScript
> -------------------------------------------
>
>                 Key: NIFI-1822
>                 URL: https://issues.apache.org/jira/browse/NIFI-1822
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Matt Burgess
>            Assignee: Matt Burgess
>             Fix For: 1.0.0, 0.7.0
>
>
> Currently ExecuteScript is annotated as TriggerSerially, meaning only one task can be running at a time. This causes issues for throughput, and can become a severe bottleneck in a flow.
> Originally this was done because the bindings for the session, context, etc. were put on a single script engine, so multiple tasks would clobber each other's bindings. However as a tradeoff (memory for capability), it would be better to have a "pool" of Script Engine instances, whose size is perhaps the max number of concurrent tasks.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)