nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roger Marin <Roger.Ma...@colesfinancialservices.com.au>
Subject NiFi operational interface
Date Mon, 03 Oct 2016 03:26:32 GMT
Hi all,

We are looking at using NiFi to replace some of our existing (batch, microbatch) Data Ingestion
& ETL/ELT processes but we have a few concerns around the "Canvas" interface.

>From a developer perspective the Canvas view is OK but we feel that it's not really suitable
from an operational point of view.  For instance, we are looking at eventually having hundreds,
perhaps even thousands of Process Groups running in production, we really need a way to easily
search and monitor any running process groups at any given time.

We were initially thinking of leveraging things like the ELK stack to build some kind of dashboard
but that doesn't give us the flexibility we need, e.g. if we need to stop a given process
group or restart it we would still need to use the canvas view for that, given this we are
looking at leveraging the NiFi API to build a new "operational" interface that we could use
to monitor and perform basic actions on running process groups (start, stop, schedule), as
well as quickly search and visualize running processors and process groups.

I am not sure if this is something that's already being worked on (I couldn't find anything
in JIRA) but I would like to know how other NiFi users in production are getting around this
(if it's even an issue for anyone else at all...).

Regards,
Roger



Mime
View raw message