nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Thomsen <mikerthom...@gmail.com>
Subject Re: Persisting/loading NIFI flows for Docker deployment
Date Wed, 08 Apr 2020 14:37:43 GMT
What I've done in the past looks something like this:

FROM apache/nifi:1.11.4
COPY flow.xml.gz /opt/nifi/nifi-current/conf/flow.xml.gz

And that's it. The obvious caveat is that you need to follow good practices
with ensuring that your flow and the way you setup the container can
replicate the environment you built it on on your host.

On Wed, Apr 8, 2020 at 10:34 AM Kevin Telford <kevin.telford@gmail.com>
wrote:

> Hi all – I have a two part question.
>
>
>
> I’d like to run NiFi inside a container in order to deploy to various
> environments. As far as I can tell, the flow.xml.gz file is the main
> “source” if you will, for a NiFi data flow.
>
> Q1) Is the flow.xml.gz file the “source” of a NiFi data flow, and if so, is
> it best practice to copy it to a new env in order to “deploy” a prebuilt
> flow? Or how best is this handled?
>
>
>
> Given that Q1 is true, my challenge then becomes somewhat Docker-specific…
>
> Situation:
>
>    - In the Dockerfile we unzip the NiFi source (L62
>    <
> https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/Dockerfile#L62
> >)
>    and then create Docker volumes (L75
>    <
> https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/Dockerfile#L75
> >
>    specifically for the conf dir). Once the container starts all the normal
>    NiFi startup things happen, and /opt/nifi/nifi-current/conf/flow.xml.gz
>    created.
>
> Complication:
>
>    - In order to persist flow.xml.gz outside of the container, I would
>    normally mount the /opt/nifi/nifi-current/conf directory, however in
> this
>    case I cannot mount it on initialization because that will overwrite
> conf
>    config files with whatever directory I bind it to (Docker container
>    isolation ensures host -> container file precedence).
>    - I could mount to a running container, but this is less ideal due to
>    the various ways a container can be deployed.
>    - I could copy manually from the running container, but this is less
>    ideal as it’s on demand, and not always persisting latest.
>
> Resolution:
>
>    - I believe instead, we would ideally create a few flow config specific
>    env vars and use them to update our nifi.properties (via
>
> https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/sh/start.sh
> ),
>    i.e. NIFI_FLOW_CONFIG_FILE_LOCATION, NIFI_FLOW_CONFIG_ARCHIVE_ENABLED,
>    NIFI_FLOW_CONFIG_ARCHIVE_DIR and so on for all nifi.flow.configuration
>    props.
>
> Q2) Would the above proposal be ideal? (add a few env vars to start.sh) –
> if so, happy to add a PR for the code and doc change. Or have others solved
> this a different way?
>
>
>
> Best,
>
> Kevin
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message