nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject Re: Rate of the data flow is exceeding the provenance recording rate. Slowing down flow to accomodate
Date Fri, 10 Jun 2016 21:02:45 GMT
What you were seeing was a warning that nifi was intentionally slowing the
flow thanks to back pressure from provenance indexing.  By switching to
volatile thee won't be any back pressure so yes you'll be faster.

Thanks
Joe
On Jun 10, 2016 4:51 PM, "Shaine Berube" <
shaine.berube@perfectsearchcorp.com> wrote:

> So will that remove the error then?  I'm really trying to remove all the
> slow downs in this data flow.
>
> On Fri, Jun 10, 2016 at 2:23 PM, Joe Witt <joe.witt@gmail.com> wrote:
>
> > Shaine,
> >
> > You can switch to using volatile/transient provenance storage.  To do
> > this edit nifi.properties
> >
> > Find this entry
> >
> > # Provenance Repository Properties
> >
> >
> nifi.provenance.repository.implementation=org.apache.nifi.provenance.PersistentProvenanceRepository
> >
> > Change the value to
> >
> > # Provenance Repository Properties
> >
> >
> nifi.provenance.repository.implementation=org.apache.nifi.provenance.VolatileProvenanceRepository
> >
> > Restart and you'll have in-memory only provenance for a very small
> > number of events.  We will add ways to more efficiently index the
> > provenance events so that even with full persistence on it will
> > support the kinds of usage patterns that can stress the current
> > approach.
> >
> > Thanks
> > Joe
> >
> > On Fri, Jun 10, 2016 at 4:19 PM, Shaine Berube
> > <shaine.berube@perfectsearchcorp.com> wrote:
> > > Any way I can stop this error?  I mean, I love the provenance
> recorder...
> > > but I don't care enough about it to slow down the data flow.
> > >
> > > --
> > > *Shaine Berube*
> >
>
>
>
> --
> *Shaine Berube*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message