nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff <jtsw...@gmail.com>
Subject Re: Processor running slow in production, not locally
Date Mon, 03 Oct 2016 15:26:06 GMT
Russel,

This sounds like it's an environmental issue.  Are you able to see the heap
usage on the production machine?  Are there enough available threads to get
the throughput you are observing when you run locally?  Have you
double-checked the scheduling tab on the processor config to make sure it
is running as aggressively as it runs locally?

I have run into this sort of thing before, and it was because of flowfile
congestion in other areas of the flow, and there were no threads available
for other processors to get through their own queues.

Just trying to think through some of the obvious/high level things that
might be affecting your flow...

- Jeff

On Mon, Oct 3, 2016 at 9:43 AM Russell Bateman <
russell.bateman@perfectsearchcorp.com> wrote:

> We use NiFi for an ETL feed. On one of the lines, we use a custom
> processor, *VelocityTemplating* (calls Apache Velocity), which work very
> well and indeed is imperceptibly fast when run locally on the same data
> (template, VTL macros, substitution fodder). However, in production it's
> another matter. What takes no time at all in local runs takes minutes in
> that environment.
>
> I'm looking for suggestions as to a) why this might be and b) how best to
> go about examining/debugging it. I think I will soon have remote-access to
> the production machine (a VPN must be set up).
>
> Thanks,
>
> Russ
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message