nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Midhun Mohan <midhun.mo...@esginc.us>
Subject Re: Queue Prioritization
Date Sun, 01 Mar 2020 12:42:05 GMT
Hi my use case is

When a invokehttp processor fails I need to stop the incoming message and
retry the failed message 2 times and if it is success start the execution
normally or if it fails again need to notify admins and stop the main
invoke http processor.



Hope you can get the scenario from this

On Sun, 1 Mar 2020, 5:30 pm Eduardo Fontes, <eduardo.fontes@gmail.com>
wrote:

> Hi Midhun,
>
> You can use DistributedMapCacheServer which starts a cache service within
> NiFi's process, but I not recommend in production. Instead you could use
> RedisConnectionPoolService.
>
> To use solution Wait/Notify the NiFi must know the number of flowfiles to
> process in one queue before start another queue. If it is a batch/on demand
> flow I think it's easier. If it is a continuos/streamming flow you must
> define a window, by number of flowfiles or by time, to switch between
> queues.
>
> Please let me know what is your case.
>
> On Fri, Feb 28, 2020 at 2:39 PM Midhun Mohan <midhun.mohan@esginc.us>
> wrote:
>
> > I think wait and notify can solve my problem. I checked the documents ,
> > which is bit unclear to me. Can anyone help me?
> >
> > Just to know few things like whether we need to configure a cache server,
> > if so which one is best
> > And also getting confused on how to configure wait and notify signals
> >
> > On Fri, 28 Feb 2020, 10:59 pm Eduardo Fontes, <eduardo.fontes@gmail.com>
> > wrote:
> >
> > > Hi! Maybe you should try another approach, like Wait/Notify or
> configure
> > > prioritizer just before the funnel, depending on your demand.
> > >
> > > On Fri, Feb 28, 2020 at 11:40 AM Midhun Mohan <midhun.mohan@esginc.us>
> > > wrote:
> > >
> > > > Hi all,
> > > > How to prioritize one queue before a funnel?
> > > >
> > > >  I need to make sure one queue is emptied before another queue is
> taken
> > > > over for executing, I tried with a priority attribute but it fails
> > when a
> > > > queue with less priority has messages before the queue with high
> > > priority.
> > > >
> > > > --
> > > >
> > > >
> > > > Regards,
> > > > Midhun Mohan
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message