Mailing-List: contact dev-help@nifi.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@nifi.apache.org
MIME-Version: 1.0
References: <CADT6bVtCXdrm1WCZJFGC4NrmZau9VL95uFNEXjRWUdknMNHcdQ@mail.gmail.com>
 <DE40A53A-6B2E-44ED-B778-D91F590F406F@gmail.com>
In-Reply-To: <DE40A53A-6B2E-44ED-B778-D91F590F406F@gmail.com>
From: Michael Kobit <mkobit@gmail.com>
Date: Thu, 21 Sep 2017 17:27:30 +0000
Message-ID: <CALELY9HzYRywYv=aYWiSDYcF+PPvSZq_ZDwM5zADVELCXoZ1CA@mail.gmail.com>
Subject: Re: Dockerfile and Docker Hub Management
To: dev@nifi.apache.org
Content-Type: multipart/alternative; boundary="94eb2c0cb4e6d5328d0559b66822"
archived-at: Thu, 21 Sep 2017 17:27:47 -0000

--94eb2c0cb4e6d5328d0559b66822
Content-Type: text/plain; charset="UTF-8"

I'd love to retire https://github.com/mkobit/docker-nifi as well since you
have started providing your own images.

On Thu, Sep 21, 2017 at 11:42 AM Kevin Doran <kdoran.apache@gmail.com>
wrote:

> Thanks, Aldrin.
>
> I'm a +1 for a separate NiFi Docker project and release process, allowing
> improvements to the containerization of NiFi artifacts to be made
> independently from the artifacts (and their source files). I have observed
> many community members maintaining their own NiFi Dockerfiles and Docker
> Compose files, and it would be nice if the most reusable variant(s) of
> these could be well maintained in one location on an independent schedule.
>
> I also agree a separate repository would be cleaner and preferable that a
> subtree of the existing NiFi repository.
>
> Thanks,
> Kevin
>
> On 9/21/17, 11:12, "Aldrin Piri" <aldrinpiri@gmail.com> wrote:
>
>     Hey folks,
>
>     ** This message turned out to be more detailed than anticipated.  In
>     summary, I propose consolidating Docker/container work with a separate
>     release process outside of the repository they are packaging.  Full
>     thoughts and background follow.  Any input would be appreciated!
>
>     ---
>
>     I've been working through providing some additional Docker
> capabilities for
>     the project and wanted to share some thoughts as well as possible
> plans to
>     help us be a bit more nimble and responsive to curating Dockerfiles and
>     their respective images on DockerHub.
>
>     As a bit of context, we currently have the core NiFi project captured
> in
>     two Dockerfiles, one that is used in conjunction with a Maven plugin
> for
>     creating an image during the NiFi build (dockermaven), and another
> that is
>     used for building tagged releases on Docker Hub (dockerhub).  Both of
> these
>     artifacts, currently, reside in a nifi-docker project and are
> activated via
>     Maven profile, (-P docker).
>
>     We've seen at times that this is a very coupled process and limits our
>     flexibility.  For instance, we had an ill-placed 'chown' which caused a
>     duplicating layer and causes our image to be doubly large.  While this
> has
>     been remedied, given current release processes, this is included with
> the
>     core nifi release and we have been unable to rectify that issue.
>
>     Another issue is a very specific sequence of actions that needs to
> happen
>     with the current release for artifacts to be triggered correctly in
> Docker
>     Hub.  This can be seen in Section 6 of the release guide [1].  While
> there
>     are ways to rectify this if the timing isn't quite right and/or an
> error is
>     made, it can impose an additional burden on the INFRA team to
> facilitate
>     these requests as there currently is no capability for PMCs to manage
> their
>     Docker repositories directly.
>
>     Ultimately, I think we should consider a separate release process for
> NiFi
>     Docker, and any associated efforts that may relate to those files.  In
> this
>     context, NiFi is encompassing of all projects/efforts in the project.
>     Additional efforts could comprise of examples of configuring NiFi to be
>     secured or clustered, receive data from MiNiFi instances, or using
> Docker
>     Compose or other orchestration frameworks. I have also noticed a
> number of
>     different areas across our work that are using Docker for integration
>     testing purposes.  With some planning and coordination, we could likely
>     consolidate many of these core resources/templates to allow us to reuse
>     them across efforts.
>
>     I believe there are two approaches from an organizational standpoint
> that
>     let us execute on the separate release process effectively:
>
>     1.) Condense all Docker artifacts into the current NiFi repository
> [2].  We
>     update our release for NiFi to exclude the Docker subtree to carry out
> our
>     normal release flow and provide the build/tooling for the Docker
> subtree to
>     be released on its own.
>
>     2.)  Establish a new git repository to handle Docker and any other
>     containerization efforts and migrate all existing resources into a file
>     structure that makes sense.
>
>     My inclination is toward (2).
>
>     Regardless of path chosen above, this frees us to handle updates and
>     improvements to container efforts when needed.  Any time we wanted to
>     release updates to Docker images, we could perform a separate release
> on
>     either the subtree of (1) or the repository of (2) and reference the
>     associated latest artifacts of NiFi.
>
>     If you've made it this far, thanks for working through the wall of
> text and
>     would appreciate any thoughts or comments.
>
>     [1] http://nifi.apache.org/release-guide.html
>     [2] https://git-wip-us.apache.org/repos/asf?p=nifi.git
>
>
>
>

--94eb2c0cb4e6d5328d0559b66822--