nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Thomsen <mikerthom...@gmail.com>
Subject Re: NiFi, Docker Environment Variable for enabling debugging of NiFi inside a Docker container
Date Thu, 21 Feb 2019 17:51:37 GMT
Erik,

I think a lot of people would benefit from having both the JVM debugger
option AND your debug logging. Go ahead and send us a PR for your debug
logging and I'll do a ticket to add the JVM debugger. I just merged a
change to the Docker setup that will make the JVM debugger much easier to
enable.

Thanks,

Mike

On Wed, Feb 20, 2019 at 9:48 AM Erik Anderson <eanders@pobox.com> wrote:

> Andrew,
>
> Sorry, was skiing.
>
> > A user only had to specify an
> > additional -debug flag for the main shell script which would in turn take
> > care of any bootstrap rewriting/generation for allowing for the remote
> JVM
> > debug session to connect.
>
> So you dont want logging, you want a remote debugger session?
>
> 1) I expect many environments wont let devs just connect a debugger into a
> production network. In a prod network, we start with logs.
> 2) What about NiFi clusters? Distributed debugging, wow, now that gives me
> a headache.  How would you connect a java debugger into a 3 node cluster?
> 3) Maybe you want to trap/breakpoint a list of exceptions and stop the
> java process to wait for a debugger to be attached?
>
> As our NiFi usage grows, the #1 most important feature is debug-ability of
> an issue in any environment.
>
> Andrew, as you have a lot more experience with issues you have seen in the
> past, how about a 30 minute call? This way can I figure out what debugging
> means to everyone, what information they need, and the best way to
> bootstrap it.
>
> Erik Anderson
> Bloomberg
> https://www.linkedin.com/in/erikanderson/
>
> Sent from my iPad
>
> > On Feb 14, 2019, at 12:47 AM, Andrew Grande <aperepel@gmail.com> wrote:
> >
> > Here's what I did previously, Erik. A user only had to specify an
> > additional -debug flag for the main shell script which would in turn take
> > care of any bootstrap rewriting/generation for allowing for the remote
> JVM
> > debug session to connect. Maybe it could give a few ideas.
> >
> > Andrew
> >
> > On Wed, Feb 13, 2019, 7:34 AM Erik Anderson <eanders@pobox.com> wrote:
> >
> >>> I was reading that email and was thinking of JVM debug options, with
> >>> suspend y/n. I guess it just shows we meant very different things by
> >> debug
> >>> mode. Maybe you could incorporate those into a PR too?
> >>>
> >>> Andrew
> >>
> >> Good point Andrew. and sorry for the slow response. I had to look at how
> >> NiFi set the JVM properties.
> >>
> >> We had issues with Java and needed to manually set env Java flags.
> >>
> >> It seems all of the Java debugging flags are set in the bootstrap.conf,
> >> located here.
> >>
> >>
> >>
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/bootstrap.conf
> >>
> >> For me, I chained together the Dockerfile
> >> ENTRYPOINT ["my_company_entry_point.sh"]
> >> then had it call the original entry point
> >> In short, all of NiFi properties are set via environment variables in a
> >> /etc/profile.d/nifi.sh on the host and propogate throughout the NiFi
> >> system. Like I listed in
> >>
> >>
> https://github.com/dprophet/nifi/blob/master/nifi-docker/dockerhub/CONFIGURATION.md
> >>
> >> I normalized all NIFI specific environment variables to start with NIFI_
> >>
> >> Why? Because you set the
> >>
> >> export NIFI_FOO="my foo"
> >>
> >> and its directly passed into the container and into the NiFi role
> account
> >> used to start the container. You now have access to the environment
> >> variables throughout the NiFi system.
> >>
> >> Example:
> >> docker run --name nifi --env-file <(env | grep NIFI_) --hostname nifi
> >>
> >> IMO, the Dockerfile entry point should allow a plug and play script so
> you
> >> can set these custom behaviors (both enterprise behaviors and custom
> >> developer/debugging). I doubt any enterprise will blindly pull a
> DockerHub
> >> container and run it. From my experience, a public container isnt
> >> enterprise friendly.
> >>
> >> Andrew, define what you want for JVM debugging, what you would want to
> set
> >> (and unset), and I will take a look.
> >>
> >> Erik Anderson
> >> Bloomberg
> >> https://www.linkedin.com/in/erikanderson/
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message