nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Payne (JIRA)" <>
Subject [jira] [Commented] (NIFI-483) Automatically elect a new primary node to allow rolling restarts
Date Thu, 02 Apr 2015 19:49:53 GMT


Mark Payne commented on NIFI-483:

There are a handful of tickets now, I think, that are fairly closely related and similar to
this one. We should probably try to organize them a bit and create an overarching ticket that
has some "children."

Probably the most important thing to do is to identify the use cases for what we're trying
to accomplish. We definitely need to update the Scheduling Strategy / primary node only to
make those separate, disparate ideas. We should be able to say CRON driven or Timer driven,
and then separately specify whether the processor is run only on the primary node. Would that
solve the use case here?

We don't currently auto-elect a new primary node specifically because if the primary node
is disconnected, the cluster manager (NCM) has no way to know whether or not the primary is
still doing its thing. It could be that the primary node died completely. Or it could be that
there's a network problem between primary node and NCM. Or maybe someone accidentally plugged
a hole in a firewall. Or any number of things. If we auto-elected a new primary node, then
we could end up with 2 different nodes doing a job that is intended to be done only by a single
node. So I guess the moral of the story is that we have to be extremely careful about changing
something like this. It's probably best to identify exactly what we are trying to accomplish
before changing something like this. But I agree, the cluster management can stand to improve.

> Automatically elect a new primary node to allow rolling restarts
> ----------------------------------------------------------------
>                 Key: NIFI-483
>                 URL:
>             Project: Apache NiFi
>          Issue Type: New Feature
>            Reporter: Ricky Saltzer
> In a clustered environment, if you restart the primary node's NiFi instance, all of the
processors scheduled on "Primary Node" will automatically be re-executed on start up. This
is a problem if you require a processor to only execute once per day, for example.  
> To get around this, you must manually promote a different node to primary before restarting
it. I think it would be a worthwhile feature to have the primary node automatically elect
another node primary if it's being restarted. 

This message was sent by Atlassian JIRA

View raw message