nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Wicks (pwicks)" <pwi...@micron.com>
Subject RE: [EXT] Re: OnPrimaryNodeStateChange vs Primary Only configuration
Date Fri, 16 Aug 2019 15:32:34 GMT
Bryan,

I'm familiar with the getNodeTypeProvider method.  Unfortunately, this does not differentiate
between processors that are scheduled to run only on the Primary node and those that are scheduled
to run on all of them.

So you're saying, a better fix would be to properly call scheduled/unscheduled, and when a
processor is unscheduled make sure it then handles this; but that it's complicated. I can
believe hat.

But, in the meantime, there probably isn't a problem with exposing this piece of scheduling
information in the ProcessContext?

Thanks,
  Peter

-----Original Message-----
From: Bryan Bende <bbende@gmail.com> 
Sent: Friday, August 16, 2019 9:19 AM
To: dev@nifi.apache.org
Subject: [EXT] Re: OnPrimaryNodeStateChange vs Primary Only configuration

AbstractSessionFactoryProcessor has a method

getNodeTypeProvider().isPrimary()

The ultimate fix for your problem is that a processor shouldn't have it's onScheduled called
at all unless it is actually schedule to run on that node. Currently it calls onScheduled
on all nodes, but then never calls onTrigger on the ones where it isn't scheduled. There is
a long standing JIRA for this, but it's a complex fix.

On Fri, Aug 16, 2019 at 11:07 AM Peter Wicks (pwicks) <pwicks@micron.com> wrote:
>
> I'm working on a bug fix for HandleHttpRequest and need to check if a processor is configured
to run only on primary node (and not if a processor has the attribute that ONLY allows it
to run on primary node).
> Here is the scenario for background:
>
>   *   NiFi cluster, but all nodes are on the same physical machine; we do this to let
developers develop/test in a cluster without needing a lot of infrastructure before deploying
to the real prod cluster.
>   *   To avoid Port conflicts, HandleHttpRequest is setup to run only on master. But,
if there is a master node change then the Http server is not properly shutdown and we get
a port conflict when the new master node starts up the new instance of the processor.
>
> The problem is I don't think the Primary Only scheduling configuration is exposed to
the processor. I'd like to do something like the code below:
>
>     @OnPrimaryNodeStateChange
>     public void onPrimaryNodeChange(final PrimaryNodeState newState) {
>         // If this processor is running in Primary Only
>         // and this is processor is not master, shutdown the http server.
>         If(this.isMasterOnlyScheduled) shutdown();
>     }
>
> I can do some work to expose this, but I thought I'd ask in case I'm missing it.
>
> Thanks,
>   Peter
Mime
View raw message