nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Gilman <matt.c.gil...@gmail.com>
Subject Re: [DISCUSS] Proposal for an Apache NiFi sub-project - NiFi Registry
Date Thu, 09 Feb 2017 13:50:12 GMT
@Joe Skora,

Definitely! This work is underway and is being tracked under this JIRA [1].

Matt

[1] https://issues.apache.org/jira/browse/NIFI-3380

On Thu, Feb 9, 2017 at 8:45 AM, Joe Skora <jskora@gmail.com> wrote:

> +1 as well.  Glad to help if needed.
>
> Do you think this will include support for versioned Processors and
> Controller Services, such that I could have SuperWidgetProcessor 1.0 and
> SuperWidgetProcessor 1.5 on the same flow?
>
> On Thu, Feb 9, 2017 at 8:42 AM, Matt Gilman <matt.c.gilman@gmail.com>
> wrote:
>
> > +1. I really like this idea. Should ease deployments between instances
> and
> > facilitate a better UX throughout the lifecycle of a dataflow.
> >
> > Matt
> >
> > On Thu, Feb 9, 2017 at 7:52 AM, Koji Kawamura <ijokarumawak@gmail.com>
> > wrote:
> >
> > > Huge +1! I was excited to read the design documents :)
> > > Agree with flow versioning at ProcessGroup level.
> > >
> > > I don't know if this is helpful, but here is an experimental project
> > > of mine which tries to achieve the same goal, versioning ProcessGroup.
> > > https://github.com/ijokarumawak/nifi-deploy-process-group
> > >
> > > It contains logics that will probably need to be implemented such as
> > > checking remaining flow files in the queues around ProcessGroup, or
> > > checking number of input/output ports from/to the ProcessGroup ... etc
> > >
> > > Hope that helps in some way, and I'd like to help make this come true!
> > >
> > > Thanks,
> > > Koji
> > >
> > > On Thu, Feb 9, 2017 at 9:43 PM, Joe Gresock <jgresock@gmail.com>
> wrote:
> > > > +1, I've been waiting for this idea since NiFi went open source!
> > > >
> > > > On Thu, Feb 9, 2017 at 4:53 AM, Ricky Saltzer <ricky@cloudera.com>
> > > wrote:
> > > >
> > > >> I'm a big +1 to this proposal. It would solve a huge burden that is
> > > keeping
> > > >> NARs up to date in environments where there's alot of teams that
> share
> > > NARs
> > > >> but have separate NiFi deployments and repositories.
> > > >>
> > > >> On Feb 8, 2017 7:09 PM, "Peter Wicks (pwicks)" <pwicks@micron.com>
> > > wrote:
> > > >>
> > > >> > I think a lot of us are facing the same challenges, and this
> sounds
> > > like
> > > >> a
> > > >> > step in the right direction.
> > > >> > I had actually started to dig into a Flow Configuration plugin
> that
> > > would
> > > >> > use Git branches to copy/sync flows between
> instances/environments,
> > > and
> > > >> > keep them versioned; hadn't gotten very far.
> > > >> >
> > > >> > -----Original Message-----
> > > >> > From: Jeremy Dyer [mailto:jdye64@gmail.com]
> > > >> > Sent: Wednesday, February 08, 2017 3:54 PM
> > > >> > To: dev@nifi.apache.org
> > > >> > Subject: Re: [DISCUSS] Proposal for an Apache NiFi sub-project
-
> > NiFi
> > > >> > Registry
> > > >> >
> > > >> > Bryan - I think this is a fantastic idea. I would also think
this
> > > would
> > > >> be
> > > >> > a good place to add a "device registry" as well. It makes much
> more
> > > sense
> > > >> > in my mind to have these efforts in sub projects outside of the
> > > >> nifi/minifi
> > > >> > core.
> > > >> >
> > > >> > On Wed, Feb 8, 2017 at 4:50 PM, Bryan Bende <bbende@gmail.com>
> > wrote:
> > > >> >
> > > >> > > NiFi Community,
> > > >> > >
> > > >> > > I'd like to initiate a discussion around creating a sub-project
> of
> > > >> > > NiFi to encompass the registry capabilities outlined in
several
> of
> > > the
> > > >> > > feature proposals on the Wiki [1]. A possible name for this
> > > >> > > sub-project is simply "NiFi Registry".
> > > >> > >
> > > >> > > Currently there are two feature proposals that call for
NiFi to
> > > >> > > interact with an external registry:
> > > >> > >
> > > >> > > Configuration Management of Flows [2]  - This feature proposal
> > calls
> > > >> > > for a flow registry where versioned flows can be published
and
> > > >> > > consumed, allowing flows to be easily migrated between
> > environments
> > > .
> > > >> > >
> > > >> > > Extension Registry [3] - This feature proposal calls for
a place
> > to
> > > >> > > publish NARs containing extensions, allowing NiFi to decouple
> > itself
> > > >> > > from including all of the NARs in the main distribution,
and
> > > allowing
> > > >> > > better discovery of available extensions.
> > > >> > >
> > > >> > > The idea would be to create a NiFi Registry sub-project,
with
> > > >> > > sub-modules for the various registries. These registries
could
> > then
> > > be
> > > >> > > packaged and distributed as a single artifact and run as
a
> > > >> > > complimentary application to NiFi and MiNiFi. NiFi would
not
> > require
> > > >> > > the registry application, however, a given NiFi could be
> > configured
> > > to
> > > >> > > know about one or more flow registries, or one or more extension
> > > >> > > registries.
> > > >> > >
> > > >> > > Creating a sub-project would allow the registry code to
evolve
> > > >> > > independently of NiFi and be released on it's own timeline.
In
> > > >> > > addition, it would make tracking issues/work much clearer
> through
> > a
> > > >> > > separate JIRA.
> > > >> > >
> > > >> > > Please discuss and provide and thoughts or feedback.
> > > >> > >
> > > >> > > Thanks,
> > > >> > >
> > > >> > > Bryan
> > > >> > >
> > > >> > > [1] https://cwiki.apache.org/confluence/display/NIFI/NiFi+
> > > >> > > Feature+Proposals
> > > >> > > [2] https://cwiki.apache.org/confluence/display/NIFI/
> > > >> > > Configuration+Management+of+Flows
> > > >> > > [3] https://cwiki.apache.org/confluence/display/NIFI/
> > > >> > > Extension+Repositories+%28aka+Extension+Registry%29+for+
> > > >> > > Dynamically-loaded+Extensions
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > I know what it is to be in need, and I know what it is to have
> > plenty.  I
> > > > have learned the secret of being content in any and every situation,
> > > > whether well fed or hungry, whether living in plenty or in want.  I
> can
> > > do
> > > > all this through him who gives me strength.    *-Philippians 4:12-13*
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message