nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan LaMothe (Jira)" <>
Subject [jira] [Created] (NIFI-7631) Create a nifi.content.repository.implementation for Apache Pulsar
Date Sat, 11 Jul 2020 18:35:00 GMT
Ryan LaMothe created NIFI-7631:

             Summary: Create a nifi.content.repository.implementation for Apache Pulsar 
                 Key: NIFI-7631
             Project: Apache NiFi
          Issue Type: New Feature
          Components: Core Framework
            Reporter: Ryan LaMothe

I would like to begin the development of a new nifi.content.repository.implementation for
Apache Pulsar. In our modern, cloud-based streaming message environments, we are using Apache
Pulsar for all of our persistent message/data and stream management. Apache NiFi currently
supports only local disk (non-volatile) and in-memory (volatile) content repository implementations.
This means that Apache NiFi currently performs double duty as both a workflow management environment
and a message/data management system, as there are no remote message/data management content
repository implementations available.

The proposed new feature development would create a new content repository implementation
designed around a streaming message/data architecture, in essence replacing the concept of
a "NiFi local queue" with an "Apache Pulsar remote queue", allowing Apache Pulsar to remotely
and independently manage messages/data on the behalf of NiFi. This would also support NiFi
as a pure workflow management environment, decoupling it from its data management responsibilities.


This message was sent by Atlassian Jira

View raw message