flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: Including option for starting job and task managers in the foreground
Date Wed, 02 Dec 2015 17:44:54 GMT
Have you looked at
https://github.com/apache/flink/tree/master/flink-contrib/docker-flink
? This demonstrates how to use Flink with Docker. In particular it
states: "Images [..] run Supervisor to stay alive when running
containers."

Have a look at flink/config-flink.sh.

Cheers,
Max

On Wed, Dec 2, 2015 at 6:29 PM, Brian Chhun
<brian.chhun@getbraintree.com> wrote:
> Yep, I think this makes sense. I'm currently patching the flink-daemon.sh
> script to remove the `&`, but I don't think it's a very robust solution,
> particularly when this script changes across versions of Flink. I'm very new
> to Docker, but the resources I've found indicates that the process must run
> in the foreground, though people seem to get around it with some hacks.
>
> When I have some time, I can look into refactoring some parts of the scripts
> so that it can be started in the foreground.
>
> Thanks,
> Brian
>
> On Wed, Dec 2, 2015 at 3:22 AM, Maximilian Michels <mxm@apache.org> wrote:
>>
>> Hi Brian,
>>
>> I don't recall Docker requires commands to run in the foreground. Still,
>> if that is your requirement, simply remove the "&" at the end of this line
>> in flink-daemon.sh:
>>
>> $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}" -classpath
>> "`manglePathList "$FLINK_TM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
>> ${CLASS_TO_RUN} "${ARGS[@]}" > "$out" 2>&1 < /dev/null &
>>
>> Cheers,
>> Max
>>
>> On Wed, Dec 2, 2015 at 9:26 AM, Till Rohrmann <trohrmann@apache.org>
>> wrote:
>>>
>>> Hi Brian,
>>>
>>> as far as I know this is at the moment not possible with our scripts.
>>> However it should be relatively easy to add by simply executing the Java
>>> command in flink-daemon.sh in the foreground. Do you want to add this?
>>>
>>> Cheers,
>>> Till
>>>
>>> On Dec 1, 2015 9:40 PM, "Brian Chhun" <brian.chhun@getbraintree.com>
>>> wrote:
>>>>
>>>> Hi All,
>>>>
>>>> Is it possible to include a command line flag for starting job and task
>>>> managers in the foreground? Currently, `bin/jobmanager.sh` and
>>>> `bin/taskmanager.sh` rely on `bin/flink-daemon.sh`, which starts these
>>>> things in the background. I'd like to execute these commands inside a docker
>>>> container, but it's expected that the process is running in the foreground.
>>>> I think it might be useful to have it run in the foreground so that it can
>>>> be hooked into some process supervisors. Any suggestions are appreciated.
>>>>
>>>>
>>>> Thanks,
>>>> Brian
>>
>>
>

Mime
View raw message