nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Thomsen <mikerthom...@gmail.com>
Subject Re: Nifi ExecuteScript slow performance
Date Mon, 13 Nov 2017 17:42:35 GMT
Vyshali,

Another trick you can do that works very well if you have a lot of
flowfiles to process is use session.get(int) to grab a batch. Obviously,
keep the volume you grab tuned to reasonable memory limits and all that,
but you can use that to make the script do a lot more work in one single
run. I have a flow that ends up processing millions of JSON files, and
grabbing 500 of them at once and merging them into a JSON array to create a
record batch really helps.

On Mon, Nov 13, 2017 at 12:23 PM, Vyshali <vyshali.n@honeywell.com> wrote:

> Thank you so much Matt.
> I will try the solutions provided and come back in case of questions.
>
> Thanks,
> Vyshali
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message