nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <ozhurakou...@hortonworks.com>
Subject Re: [DISCUSS] Proposal for an Apache NiFi sub-project - NiFi Registry
Date Thu, 09 Feb 2017 14:19:36 GMT
Bryan

While I am huge +1 for breaking up NiFi into manageable structure, I am still unclear as to
the scope of this project.

1. As discussed in [3], there are public repositories and services out there that have been
designed for the exact problem we are facing in NiFi today. Those repositories and services
(i.e., Bintray) have been embraced by the larger OS community of developers and open source
projects already. So at the very least we need to have some answer/story about why we decided
NOT to rely on them (if that is the decision).
2. On the flip side, it is perfectly valid to assume that someone or some organization will
not find solutions such as BinTray attractive or usable in their environment and may prefer
a custom solution. In that case having alternatives may help, but in itself is not a solution
since all of the available alternatives may also NOT be sufficient. 
So, in other words relying on any existing solution or building multiple may not solve the
issue I (the customer) may have. This leaves only one option - a project that defines a set
of strategies (interfaces) that would allow one to either integrate with an already existing
solution or implement a new custom one. Now, we (NiFi community) could contribute integration
modules to such project based on those pre-defined Registry interfaces. IMHO such approach
will foster more community collaboration and acceptance by exposing such integration model.

Also, regardless of the decision, the bigger work will be overhauling NiFi deployment/packaging
model and UI interaction since with external registries NARs, flows etc., would need to be
pulled/removed/updated while NiFi is running and that I think is a much bigger work.

In the end I am +1 for the project and super excited that the idea is finally starting to
get traction, but wanted to contribute with above comments to ensure that the project's scope
is clearly defined.

Cheers
Oleg
> On Feb 8, 2017, at 4:50 PM, Bryan Bende <bbende@gmail.com> wrote:
> 
> NiFi Community,
> 
> I'd like to initiate a discussion around creating a sub-project of
> NiFi to encompass the registry capabilities outlined in several of the
> feature proposals on the Wiki [1]. A possible name for this
> sub-project is simply "NiFi Registry".
> 
> Currently there are two feature proposals that call for NiFi to
> interact with an external registry:
> 
> Configuration Management of Flows [2]  - This feature proposal calls
> for a flow registry where versioned flows can be published and
> consumed, allowing flows to be easily migrated between environments .
> 
> Extension Registry [3] - This feature proposal calls for a place to
> publish NARs containing extensions, allowing NiFi to decouple itself
> from including all of the NARs in the main distribution, and allowing
> better discovery of available extensions.
> 
> The idea would be to create a NiFi Registry sub-project, with
> sub-modules for the various registries. These registries could then be
> packaged and distributed as a single artifact and run as a
> complimentary application to NiFi and MiNiFi. NiFi would not require
> the registry application, however, a given NiFi could be configured to
> know about one or more flow registries, or one or more extension
> registries.
> 
> Creating a sub-project would allow the registry code to evolve
> independently of NiFi and be released on it's own timeline. In
> addition, it would make tracking issues/work much clearer through a
> separate JIRA.
> 
> Please discuss and provide and thoughts or feedback.
> 
> Thanks,
> 
> Bryan
> 
> [1] https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals
> [2] https://cwiki.apache.org/confluence/display/NIFI/Configuration+Management+of+Flows
> [3] https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
> 


Mime
View raw message