nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Doran <kdo...@apache.org>
Subject Re: [DISCUSS] Next version of NiFi Registry after 0.5.0
Date Wed, 04 Sep 2019 17:21:08 GMT
Hi Bryan and all,

Thanks for starting this discussion thread; good stuff.

I am also leaning towards option #3 (master becomes 1.0.0-SNAPSHOT,
but we do not raise the minimum required Java version at this time). I
would be fine with raising the major version to 2.x again when we do
make Java 11 a minimum requirement, presumably sometime in the next
year or so.

I don't have any suggestions for breaking changes that could take
advantage of a major version change (other than the ones Bryan
presented), that I feel strongly about, but with discussion / dev work
starting on new major versions, it might be good to start tracking
issues in Jira that specifically would require a major version change.
That way, if someone has time, those items could be tracked extra
closely anytime we are developing towards a new major version.

I propose something like a `majorVersion` or `breakingChange` label to
use while those items are in the backlog, until wok starts/lands and
then the Fix Version field can be used to specify the exact version it
will be included in.

Thanks,
Kevin






On Wed, Aug 28, 2019 at 6:56 PM Craig Knell <craig.knell@gmail.com> wrote:
>
> Hi folks
>
> I’m not sure where the functionality resides but could we include rotating certificates
>
> Best regards
>
> Craig
>
> > On 28 Aug 2019, at 21:41, Pierre Villard <pierre.villard.fr@gmail.com> wrote:
> >
> > Thanks for the recap Bryan. I'm a +1 on option 3.
> >
> > Pierre
> >
> >> Le mer. 28 août 2019 à 15:35, Bryan Bende <bbende@gmail.com> a écrit
:
> >>
> >> I think Pierre summed exactly what I was trying to get at. I realize
> >> now that I did a poor job of explaining, but it seemed like there were
> >> three approaches we could take...
> >>
> >> 1) Master becomes 0.6.0-SNAPSHOT and we continue on like normal. My HA
> >> work has to sit somewhere indefinitely, either a draft PR or a branch,
> >> and it likely ends up getting harder and harder to keep in sync with
> >> master as time goes on.
> >>
> >> 2) Master becomes 1.0.0-SNAPSHOT, the HA work I'm doing can be merged
> >> whenever its ready, and we wait until we make Java 11 the min
> >> requirement to make the 1.0.0 release, but this could take who knows
> >> how long (6 months, 12 months ?) and there would be no releases during
> >> that time.
> >>
> >> 3) Master becomes 1.0.0-SNAPSHOT, the HA work I'm doing can be merged
> >> whenever its ready, we make the 1.0.0 release whenever its ready with
> >> Java 8/9/11 support, and then later we do a 2.0.0 release for Java 11
> >> min requirement, we just might have some short live 1.x line, but that
> >> shouldn't really matter.
> >>
> >> I think #3 is probably the most realistic option.
> >>
> >> All of the other ideas suggested are good items to keep in mind, and
> >> definitely represent valuable functionality, but as far as I know,
> >> they could be implemented on any second digit release within 0.x or
> >> 1.x, its really the breaking changes I was trying to identify that
> >> would warrant going to a third digit release.
> >>
> >>
> >> On Wed, Aug 28, 2019 at 4:56 AM Pierre Villard
> >> <pierre.villard.fr@gmail.com> wrote:
> >>>
> >>> If we believe that NiFi 2.x will only support Java 11+, should we do the
> >>> same for NiFi Registry 1.x? It'd probably be more consistent for users,
> >> no?
> >>> Or... if we think that NiFi 2.x is not going to happen soon, we could
> >> have
> >>> a "short lived" Registry 1.x branch with Java 8/9/11 support, and then
> >> have
> >>> Registry 2.x when we go for Java 11+.
> >>>
> >>> Other than that, strong +1 for moving to a 1.x branch and have the new
> >>> features with the API changes.
> >>>
> >>> Le mar. 27 août 2019 à 23:16, Andy LoPresto <alopresto@apache.org>
a
> >> écrit :
> >>>
> >>>> I am definitely a +1 on moving to 1.0.0-SNAPSHOT and allowing the API
> >> to
> >>>> change to support the new functionality.
> >>>>
> >>>> Andy LoPresto
> >>>> alopresto@apache.org
> >>>> alopresto.apache@gmail.com
> >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>>>
> >>>>> On Aug 27, 2019, at 3:02 PM, Bryan Bende <bbende@gmail.com>
wrote:
> >>>>>
> >>>>> Evan,
> >>>>>
> >>>>> Thanks for the input... definitely some improvements around merging
> >>>>> and concurrent modifications that can be made, although we have
to
> >>>>> figure out which parts of these are actually in NiFi Registry code
vs
> >>>>> NiFi code. Many times a lot of this logic is implemented on NiFi
> >> side.
> >>>>>
> >>>>> Regarding your feedback about controller services, this will be
> >>>>> resolved in NiFi 1.10.0 + Registry 0.5.0 :) it will now auto-resolve
> >>>>> services by name from a parent group, as long as there is only one
> >>>>> service with that name and type (name is not unique in NiFi).
> >>>>>
> >>>>>
> >>>>> Mike,
> >>>>>
> >>>>> Great points. I think we need to divide extension registry into
two
> >>>>> different things...
> >>>>>
> >>>>> 1) General functionality to support versioned extensions
> >>>>> 2) The centrally hosted extension registry for the Apache community.
> >>>>>
> >>>>> The reason I say this is because #2 has a whole set of separate
> >> things
> >>>>> to figure out like where is it going to be hosted, who is going
to
> >> pay
> >>>>> for it, how is the community going to manage releases of NARs
> >>>>> (restructuring of repos), etc, and while all of that is very
> >>>>> important, I don't think it is really related to whether or not
we
> >>>>> would release a specific version of NiFi Registry.
> >>>>>
> >>>>> For #1, most of the work that is needed on NiFi Registry side is
> >>>>> already done.  We need the NiFi 1.10.0 release which provides the
> >>>>> changes to nifi-api that allow a NAR to be uploaded to Registry
> >>>>> 0.4.0/0.5.0, and we also need the CLI from 1.10.0 which provides
> >>>>> commands for making it easy to upload NARs from scripts.
> >>>>>
> >>>>> The missing pieces are more on the NiFi side of things... stuff
like
> >>>>> how does someone install/manage extension bundles from the NiFi
UI or
> >>>>> REST API, how does NiFi import a versioned flow and automatically
> >>>>> install the missing bundles, etc. These things would all be
> >>>>> implemented in NiFi.
> >>>>>
> >>>>> For pushing builds, we can definitely look at integration with Nexus,
> >>>>> and I was also thinking about having another Maven plugin like
> >>>>> "nifi-registry-maven-plugin" which would release your NAR to registry
> >>>>> as part of a given Maven lifecycle.
> >>>>>
> >>>>>
> >>>>> On Tue, Aug 27, 2019 at 2:28 PM Evan Reynolds
> >> <evan@usermind.com.invalid>
> >>>> wrote:
> >>>>>>
> >>>>>> I'd love to suggest working on improvements to avoid merge
> >> conflicts,
> >>>> and also more intelligence in connecting things like controller
> >> services.
> >>>>>>
> >>>>>> Merge conflicts:
> >>>>>> We are using the registry to deploy flows to production. We
have
> >>>> conflicts far too frequently in the pipeline, and as there is no way
to
> >>>> merge we are left manually replicating one set of changes over the
> >> other
> >>>> set of changes, then roll that set back.
> >>>>>>
> >>>>>> We have been working on processes to avoid that, but I do not
think
> >>>> we're ever going to fully succeed. __ I would love to have a
> >> discussion of
> >>>> how to avoid that! I know there aren't easy fixes (we can merge the
XML
> >>>> files but how to prove that is actually the desired result would be
a
> >>>> nightmare) but how many hooks does the registry have in NiFi? Could
it
> >>>> monitor the deployed flows, for example, and if one flow changed could
> >> it
> >>>> lock the other deployments so that if someone tried to change them
> >> they'd
> >>>> get notified that they were about to create a problem so that they
> >> could
> >>>> avoid it?
> >>>>>>
> >>>>>> I'm not saying that is a good plan. But it's there to start
a
> >>>> discussion mostly!
> >>>>>>
> >>>>>> Controller services:
> >>>>>> I have a controller service that I can only have one of due
to
> >> resource
> >>>> limitations, but several flows use it. When I deploy a flow, since the
> >>>> controller service is not inside that flow, it doesn't deploy with that
> >>>> connection hooked up correctly so I have to go fix several places. I'd
> >> love
> >>>> to improve that somehow!
> >>>>>>
> >>>>>> ---------------------------------------------------------------
> >>>>>>
> >>>>>> Evan Reynolds
> >>>>>> evan@usermind.com
> >>>>>>
> >>>>>>
> >>>>>> On 8/26/19, 2:56 PM, "Mike Thomsen" <mikerthomsen@gmail.com>
> >> wrote:
> >>>>>>
> >>>>>>   I'd like to see the extension registry at the top of the list,
as
> >>>> well as
> >>>>>>   discussions about what sort of workflows are envisioned to
make
> >> the
> >>>>>>   Registry useful for DevOps teams. For example, would we want
to
> >> have
> >>>>>>   integration with Nexus and similar tools to allow a CI/CD
tool to
> >>>> push
> >>>>>>   builds into the central repository and let the Registry pull
them
> >>>> down or
> >>>>>>   should we go for pushing directly to the Registry?
> >>>>>>
> >>>>>>   On Mon, Aug 26, 2019 at 11:42 AM Bryan Bende <bbende@gmail.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Typically after a release we set master to the next second
digit
> >>>>>>> version (i.e. release 0.5.0 and then master goes to
> >> 0.6.0-SNAPSHOT),
> >>>>>>> but I wanted to discuss the idea of working towards a 1.0.0
release
> >>>>>>> for NiFi Registry.
> >>>>>>>
> >>>>>>> I've been doing some work to support an HA deployment of
NiFi
> >> Registry
> >>>>>>> and it requires breaking changes to the REST API to introduce
an
> >>>>>>> optimistic locking strategy. I'd like to be able to land
this work
> >> in
> >>>>>>> master in the near future.
> >>>>>>>
> >>>>>>> I wanted to see what other work/changes we might want to
target
> >> for a
> >>>>>>> 1.0.0 release. One big item would be Java 11 support, but
> >> technically
> >>>>>>> we can do that without a major release (just like we are
doing in
> >>>>>>> NiFi). The question would be whether we want NiFi Registry
1.0.0 to
> >>>>>>> make Java 11 the minimum, and if so, then at what point
would we be
> >>>>>>> ready to do that.
> >>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>

Mime
View raw message