flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Chhun <brian.ch...@getbraintree.com>
Subject Re: Including option for starting job and task managers in the foreground
Date Wed, 02 Dec 2015 18:16:41 GMT
Thanks, I'm basing the things I'm doing based on what I see there. One
thing that's not clear to me in that example is why supervisor is used to
keep the container alive, rather than using some simpler means. It doesn't
look like it's been configured to supervise anything.

On Wed, Dec 2, 2015 at 11:44 AM, Maximilian Michels <mxm@apache.org> wrote:

> Have you looked at
> https://github.com/apache/flink/tree/master/flink-contrib/docker-flink
> ? This demonstrates how to use Flink with Docker. In particular it
> states: "Images [..] run Supervisor to stay alive when running
> containers."
>
> Have a look at flink/config-flink.sh.
>
> Cheers,
> Max
>
> On Wed, Dec 2, 2015 at 6:29 PM, Brian Chhun
> <brian.chhun@getbraintree.com> wrote:
> > Yep, I think this makes sense. I'm currently patching the flink-daemon.sh
> > script to remove the `&`, but I don't think it's a very robust solution,
> > particularly when this script changes across versions of Flink. I'm very
> new
> > to Docker, but the resources I've found indicates that the process must
> run
> > in the foreground, though people seem to get around it with some hacks.
> >
> > When I have some time, I can look into refactoring some parts of the
> scripts
> > so that it can be started in the foreground.
> >
> > Thanks,
> > Brian
> >
> > On Wed, Dec 2, 2015 at 3:22 AM, Maximilian Michels <mxm@apache.org>
> wrote:
> >>
> >> Hi Brian,
> >>
> >> I don't recall Docker requires commands to run in the foreground. Still,
> >> if that is your requirement, simply remove the "&" at the end of this
> line
> >> in flink-daemon.sh:
> >>
> >> $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}"
> -classpath
> >> "`manglePathList "$FLINK_TM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
> >> ${CLASS_TO_RUN} "${ARGS[@]}" > "$out" 2>&1 < /dev/null &
> >>
> >> Cheers,
> >> Max
> >>
> >> On Wed, Dec 2, 2015 at 9:26 AM, Till Rohrmann <trohrmann@apache.org>
> >> wrote:
> >>>
> >>> Hi Brian,
> >>>
> >>> as far as I know this is at the moment not possible with our scripts.
> >>> However it should be relatively easy to add by simply executing the
> Java
> >>> command in flink-daemon.sh in the foreground. Do you want to add this?
> >>>
> >>> Cheers,
> >>> Till
> >>>
> >>> On Dec 1, 2015 9:40 PM, "Brian Chhun" <brian.chhun@getbraintree.com>
> >>> wrote:
> >>>>
> >>>> Hi All,
> >>>>
> >>>> Is it possible to include a command line flag for starting job and
> task
> >>>> managers in the foreground? Currently, `bin/jobmanager.sh` and
> >>>> `bin/taskmanager.sh` rely on `bin/flink-daemon.sh`, which starts these
> >>>> things in the background. I'd like to execute these commands inside
a
> docker
> >>>> container, but it's expected that the process is running in the
> foreground.
> >>>> I think it might be useful to have it run in the foreground so that
> it can
> >>>> be hooked into some process supervisors. Any suggestions are
> appreciated.
> >>>>
> >>>>
> >>>> Thanks,
> >>>> Brian
> >>
> >>
> >
>

Mime
View raw message