nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Kobit <>
Subject Re: Dockerfile and Docker Hub Management
Date Thu, 21 Sep 2017 17:27:30 GMT
I'd love to retire as well since you
have started providing your own images.

On Thu, Sep 21, 2017 at 11:42 AM Kevin Doran <>

> Thanks, Aldrin.
> I'm a +1 for a separate NiFi Docker project and release process, allowing
> improvements to the containerization of NiFi artifacts to be made
> independently from the artifacts (and their source files). I have observed
> many community members maintaining their own NiFi Dockerfiles and Docker
> Compose files, and it would be nice if the most reusable variant(s) of
> these could be well maintained in one location on an independent schedule.
> I also agree a separate repository would be cleaner and preferable that a
> subtree of the existing NiFi repository.
> Thanks,
> Kevin
> On 9/21/17, 11:12, "Aldrin Piri" <> wrote:
>     Hey folks,
>     ** This message turned out to be more detailed than anticipated.  In
>     summary, I propose consolidating Docker/container work with a separate
>     release process outside of the repository they are packaging.  Full
>     thoughts and background follow.  Any input would be appreciated!
>     ---
>     I've been working through providing some additional Docker
> capabilities for
>     the project and wanted to share some thoughts as well as possible
> plans to
>     help us be a bit more nimble and responsive to curating Dockerfiles and
>     their respective images on DockerHub.
>     As a bit of context, we currently have the core NiFi project captured
> in
>     two Dockerfiles, one that is used in conjunction with a Maven plugin
> for
>     creating an image during the NiFi build (dockermaven), and another
> that is
>     used for building tagged releases on Docker Hub (dockerhub).  Both of
> these
>     artifacts, currently, reside in a nifi-docker project and are
> activated via
>     Maven profile, (-P docker).
>     We've seen at times that this is a very coupled process and limits our
>     flexibility.  For instance, we had an ill-placed 'chown' which caused a
>     duplicating layer and causes our image to be doubly large.  While this
> has
>     been remedied, given current release processes, this is included with
> the
>     core nifi release and we have been unable to rectify that issue.
>     Another issue is a very specific sequence of actions that needs to
> happen
>     with the current release for artifacts to be triggered correctly in
> Docker
>     Hub.  This can be seen in Section 6 of the release guide [1].  While
> there
>     are ways to rectify this if the timing isn't quite right and/or an
> error is
>     made, it can impose an additional burden on the INFRA team to
> facilitate
>     these requests as there currently is no capability for PMCs to manage
> their
>     Docker repositories directly.
>     Ultimately, I think we should consider a separate release process for
> NiFi
>     Docker, and any associated efforts that may relate to those files.  In
> this
>     context, NiFi is encompassing of all projects/efforts in the project.
>     Additional efforts could comprise of examples of configuring NiFi to be
>     secured or clustered, receive data from MiNiFi instances, or using
> Docker
>     Compose or other orchestration frameworks. I have also noticed a
> number of
>     different areas across our work that are using Docker for integration
>     testing purposes.  With some planning and coordination, we could likely
>     consolidate many of these core resources/templates to allow us to reuse
>     them across efforts.
>     I believe there are two approaches from an organizational standpoint
> that
>     let us execute on the separate release process effectively:
>     1.) Condense all Docker artifacts into the current NiFi repository
> [2].  We
>     update our release for NiFi to exclude the Docker subtree to carry out
> our
>     normal release flow and provide the build/tooling for the Docker
> subtree to
>     be released on its own.
>     2.)  Establish a new git repository to handle Docker and any other
>     containerization efforts and migrate all existing resources into a file
>     structure that makes sense.
>     My inclination is toward (2).
>     Regardless of path chosen above, this frees us to handle updates and
>     improvements to container efforts when needed.  Any time we wanted to
>     release updates to Docker images, we could perform a separate release
> on
>     either the subtree of (1) or the repository of (2) and reference the
>     associated latest artifacts of NiFi.
>     If you've made it this far, thanks for working through the wall of
> text and
>     would appreciate any thoughts or comments.
>     [1]
>     [2]

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message