nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eduardo Fontes <eduardo.fon...@gmail.com>
Subject Re: Queue Prioritization
Date Sun, 01 Mar 2020 12:00:15 GMT
Hi Midhun,

You can use DistributedMapCacheServer which starts a cache service within
NiFi's process, but I not recommend in production. Instead you could use
RedisConnectionPoolService.

To use solution Wait/Notify the NiFi must know the number of flowfiles to
process in one queue before start another queue. If it is a batch/on demand
flow I think it's easier. If it is a continuos/streamming flow you must
define a window, by number of flowfiles or by time, to switch between
queues.

Please let me know what is your case.

On Fri, Feb 28, 2020 at 2:39 PM Midhun Mohan <midhun.mohan@esginc.us> wrote:

> I think wait and notify can solve my problem. I checked the documents ,
> which is bit unclear to me. Can anyone help me?
>
> Just to know few things like whether we need to configure a cache server,
> if so which one is best
> And also getting confused on how to configure wait and notify signals
>
> On Fri, 28 Feb 2020, 10:59 pm Eduardo Fontes, <eduardo.fontes@gmail.com>
> wrote:
>
> > Hi! Maybe you should try another approach, like Wait/Notify or configure
> > prioritizer just before the funnel, depending on your demand.
> >
> > On Fri, Feb 28, 2020 at 11:40 AM Midhun Mohan <midhun.mohan@esginc.us>
> > wrote:
> >
> > > Hi all,
> > > How to prioritize one queue before a funnel?
> > >
> > >  I need to make sure one queue is emptied before another queue is taken
> > > over for executing, I tried with a priority attribute but it fails
> when a
> > > queue with less priority has messages before the queue with high
> > priority.
> > >
> > > --
> > >
> > >
> > > Regards,
> > > Midhun Mohan
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message