nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre Villard <pierre.villard...@gmail.com>
Subject Re: validation changes in 1.7.0
Date Tue, 03 Jul 2018 07:59:00 GMT
Hey Mark,

I agree with the team here and suspect it could be related to EL: the
changes I did for the Expression Language scope can cause existing unit
tests to fail with NiFi 1.7.0. If a property is supporting expression
language without defining the scope (1.6.0 and below), the scope defaults
to VARIABLE_REGISTRY in 1.7.0. However, if flow files are used to evaluate
the expression language during a unit test, it'll fail saying that the
scope is wrong: scope should be FLOWFILE_ATTRIBUTES instead of
VARIABLE_REGISTRY. This failure is on purpose to ensure the proper scope is
set on the properties based on how properties are used in the processor.

Note that, if that's indeed the issue, only the unit tests are impacted by
the changes, processor behavior won't change after the upgrade to 1.7.0.

Pierre


2018-07-03 0:17 GMT+02:00 Mike Thomsen <mikerthomsen@gmail.com>:

> Mark,
>
> Matt raises a very good point there. Please share the stack traces because
> if they follow a general pattern of complaining "it was scoped for this,
> but not for that," we can probably get you up and running again very
> quickly.
>
> For what it's worth, from my own testing, this is entirely a testing
> framework issue. The Mongo bundle was blowing up with all sorts of false
> errors during that time, and yet it never seemed to affect the processors
> once they were actually running.
>
> On Mon, Jul 2, 2018 at 5:26 PM Matt Burgess <mattyb149@gmail.com> wrote:
>
> > Do your processors work against 1.6.0? I’m wondering if there’s something
> > with the new Expression Language scope stuff, I believe that can cause
> unit
> > test failures even if it still works (albeit deprecated) on a live NiFi.
> >
> > Regards,
> > Matt
> >
> > > On Jul 2, 2018, at 4:52 PM, Mark Payne <markap14@hotmail.com> wrote:
> > >
> > > Hey Mark,
> > >
> > > The validation logic was changed so that instead of performing
> > validation on demand each time that
> > > the user refreshes stats, navigates to a process group, etc., it all is
> > done asynchronously. We then periodically
> > > perform validation in the background.
> > >
> > > So the idea is that we changed when validation is called, not how it is
> > called. So I'm not sure why you would have
> > > seen customValidate() being called previously but not anymore. It is
> > possible that there was previously an implicit
> > > call (implicit from the perspective of the unit test, not from the
> > perspective of the Test Runner) to validate a
> > > component that got removed in some of this refactoring.
> > >
> > > I do see in StandardProcessorTestRunner (the impl of the TestRunner
> > interface) that when run() is called, are still
> > > performing validation as we were previously and asserting that the
> > processor is valid. It's possible, though, that
> > > there is another code path where we now fail to call validate()? Can
> you
> > share any more information about what
> > > your unit tests are doing, that previously worked but no longer do?
> > >
> > > Thanks
> > > -Mark
> > >
> > >
> > >> On Jul 2, 2018, at 3:59 PM, Mark Bean <mark.o.bean@gmail.com> wrote:
> > >>
> > >> Several of our custom processors are not compatible with version
> 1.7.0.
> > >> There is a growing pattern of unit test failures due to
> customValidate()
> > >> not being executed now where it was being executed in 1.6.0 and
> prior. I
> > >> believe there were changes made in 1.7.0 in terms of how validation -
> > and
> > >> therefore customValidate() - is performed and/or when it is performed.
> > >>
> > >> Could someone provide a brief description on how validation was
> changed
> > in
> > >> 1.7.0?
> > >>
> > >> Thanks!
> > >> Mark
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message