nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: Opinions on Assembly target location
Date Wed, 27 Feb 2019 17:52:23 GMT
I think this situation is sort of analogous to some other scenarios we
have run into, such as the integrations between NiFi and stream
processing.

For example, the storm and spark integration lives in NiFi's code
base, and NiFi is responsible for determining if a new version of
those APIs is available, updating the dependency, and resolving any
necessary code changes.

The Flink and Apex integration lives in each of those respective code
bases, so if Flink changes all of their APIs, they will be forced to
update the NiFi Flink source/sink in order to make their build pass.

I can see value in either approach. If we consider the JNI stuff to be
a more generic capability, then I think I lean towards NiFi owning it.

Even with that, MiNiFi will still be dependent on some version of
whatever JNI artifacts NiFi produces, so MiNiFi would still have
ownership over when to change that dependency and ensure that it still
works properly in a given release of MiNiFi.

On Wed, Feb 27, 2019 at 12:02 PM Arpad Boda <aboda@hortonworks.com> wrote:
>
> Hi,
>
> In my opinion, there is another question here to be answered: what version of NiFi NARs
do we expect to work in MiNiFi?
>
> I see two different approaches here:
> -#1) MiNiFi is compatible with a given NiFi version, like 1.9.0, NARs in that release
work properly in MiNiFi.
> In this case the tests belong to MiNiFi and NiFi there becomes a dependency -> the
tests are required to pass when upgrading the dependency.
> -#2) MiNiFi is compatible with master. This means that whatever we introduce in NiFi
shouldn't break, so the tests more belong to NiFi.
>
> According to my experience preventing the merge of a breaking change is easier than fixing
later, which would make me vote for #2, however this approach has a side effect, too: introducing
a breaking change (that MiNiFi needs to adapt) becomes a huge pain, it's much easier in #1.
>
> Regards,
> Arpad
>
> ´╗┐On 27/02/2019, 17:44, "Marc Parisi" <phrocker@apache.org> wrote:
>
>     Hi ,
>        I've submitted a PR to Apache NiFi MiNiFi C++ that enables the project
>     to access and run NiFi processors. This requires that certain jars be
>     included to bootstrap that process long with a set of NARs from which we
>     will access the desired components. Effectively this means that a user can
>     run the current set of C++ processors alongside Java processors.
>        To facilitate this feature's use, I submitted a PR [1] to NiFi with that
>     assembles the appropriate JARs and a basic set of useful/necessary NARs.
>     I'd like opinions on where this should live. The options have been to place
>     it within Apache NiFi MiNIFi C++ ( specifically within the aforementioned
>     feature's build directory) as it only relates to that project and, perhaps
>     allowing us to more easily address problems...or as per the PR: placing it
>     in NiFi proper, hopefully building regression tests that notify us of
>     issues before a release.
>
>        I've been wavering on the options and want community opinion. The end
>     goal in either case is that if/when a user builds with this functionality
>     we can deliver a package that runs out of the box. This can be done with
>     either approach.
>
>         The net positive of placing it in NiFi as I see it: sense that it is an
>     official feature and perhaps notification before NiFi release or in
>     travis.  Side effects include: questioning if this belongs here, hair loss,
>     and perhaps a sudden loss of appetite.
>
>         The net positive of isolating this within MiNIFi C++: cleaner end
>     solution, it's produced in the same place it's consumed, ability to notify
>     of problems in our CI and release process ( this could be viewed as a
>     negative too ? ).
>
>     TL;DR:  Where feature Live?
>
>         Any insight/opinions are appreciated. Thanks!
>
>     [1] https://github.com/apache/nifi/pull/3330
>
>

Mime
View raw message