nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy LoPresto <alopre...@apache.org>
Subject Re: [DISCUSS] Next version of NiFi Registry after 0.5.0
Date Tue, 27 Aug 2019 21:16:10 GMT
I am definitely a +1 on moving to 1.0.0-SNAPSHOT and allowing the API to change to support
the new functionality. 

Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Aug 27, 2019, at 3:02 PM, Bryan Bende <bbende@gmail.com> wrote:
> 
> Evan,
> 
> Thanks for the input... definitely some improvements around merging
> and concurrent modifications that can be made, although we have to
> figure out which parts of these are actually in NiFi Registry code vs
> NiFi code. Many times a lot of this logic is implemented on NiFi side.
> 
> Regarding your feedback about controller services, this will be
> resolved in NiFi 1.10.0 + Registry 0.5.0 :) it will now auto-resolve
> services by name from a parent group, as long as there is only one
> service with that name and type (name is not unique in NiFi).
> 
> 
> Mike,
> 
> Great points. I think we need to divide extension registry into two
> different things...
> 
> 1) General functionality to support versioned extensions
> 2) The centrally hosted extension registry for the Apache community.
> 
> The reason I say this is because #2 has a whole set of separate things
> to figure out like where is it going to be hosted, who is going to pay
> for it, how is the community going to manage releases of NARs
> (restructuring of repos), etc, and while all of that is very
> important, I don't think it is really related to whether or not we
> would release a specific version of NiFi Registry.
> 
> For #1, most of the work that is needed on NiFi Registry side is
> already done.  We need the NiFi 1.10.0 release which provides the
> changes to nifi-api that allow a NAR to be uploaded to Registry
> 0.4.0/0.5.0, and we also need the CLI from 1.10.0 which provides
> commands for making it easy to upload NARs from scripts.
> 
> The missing pieces are more on the NiFi side of things... stuff like
> how does someone install/manage extension bundles from the NiFi UI or
> REST API, how does NiFi import a versioned flow and automatically
> install the missing bundles, etc. These things would all be
> implemented in NiFi.
> 
> For pushing builds, we can definitely look at integration with Nexus,
> and I was also thinking about having another Maven plugin like
> "nifi-registry-maven-plugin" which would release your NAR to registry
> as part of a given Maven lifecycle.
> 
> 
> On Tue, Aug 27, 2019 at 2:28 PM Evan Reynolds <evan@usermind.com.invalid> wrote:
>> 
>> I'd love to suggest working on improvements to avoid merge conflicts, and also more
intelligence in connecting things like controller services.
>> 
>> Merge conflicts:
>> We are using the registry to deploy flows to production. We have conflicts far too
frequently in the pipeline, and as there is no way to merge we are left manually replicating
one set of changes over the other set of changes, then roll that set back.
>> 
>> We have been working on processes to avoid that, but I do not think we're ever going
to fully succeed. __ I would love to have a discussion of how to avoid that! I know there
aren't easy fixes (we can merge the XML files but how to prove that is actually the desired
result would be a nightmare) but how many hooks does the registry have in NiFi? Could it monitor
the deployed flows, for example, and if one flow changed could it lock the other deployments
so that if someone tried to change them they'd get notified that they were about to create
a problem so that they could avoid it?
>> 
>> I'm not saying that is a good plan. But it's there to start a discussion mostly!
>> 
>> Controller services:
>> I have a controller service that I can only have one of due to resource limitations,
but several flows use it. When I deploy a flow, since the controller service is not inside
that flow, it doesn't deploy with that connection hooked up correctly so I have to go fix
several places. I'd love to improve that somehow!
>> 
>> ---------------------------------------------------------------
>> 
>> Evan Reynolds
>> evan@usermind.com
>> 
>> 
>> ´╗┐On 8/26/19, 2:56 PM, "Mike Thomsen" <mikerthomsen@gmail.com> wrote:
>> 
>>    I'd like to see the extension registry at the top of the list, as well as
>>    discussions about what sort of workflows are envisioned to make the
>>    Registry useful for DevOps teams. For example, would we want to have
>>    integration with Nexus and similar tools to allow a CI/CD tool to push
>>    builds into the central repository and let the Registry pull them down or
>>    should we go for pushing directly to the Registry?
>> 
>>    On Mon, Aug 26, 2019 at 11:42 AM Bryan Bende <bbende@gmail.com> wrote:
>> 
>>> Typically after a release we set master to the next second digit
>>> version (i.e. release 0.5.0 and then master goes to 0.6.0-SNAPSHOT),
>>> but I wanted to discuss the idea of working towards a 1.0.0 release
>>> for NiFi Registry.
>>> 
>>> I've been doing some work to support an HA deployment of NiFi Registry
>>> and it requires breaking changes to the REST API to introduce an
>>> optimistic locking strategy. I'd like to be able to land this work in
>>> master in the near future.
>>> 
>>> I wanted to see what other work/changes we might want to target for a
>>> 1.0.0 release. One big item would be Java 11 support, but technically
>>> we can do that without a major release (just like we are doing in
>>> NiFi). The question would be whether we want NiFi Registry 1.0.0 to
>>> make Java 11 the minimum, and if so, then at what point would we be
>>> ready to do that.
>>> 
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message