flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: Job Manager HA manual setup
Date Sun, 28 Feb 2016 15:10:44 GMT
Hey Welly!

Yes, it is possible to do manually via the jobmanager.sh and
taskmanager.sh script like this:

jobmanager.sh start cluster $HOST $WEB-UI-PORT
taskmanager.sh start

The start-cluster.sh script is just a wrapper around these scripts.

>From experience, it's often forgotten to sync the configuration files.
Make sure to have the same configuration file on each host (both job
and task managers), because that is relevant to parse the ZooKeeper
quorum etc.

The task managers retrieve the currently leading job manager via
ZooKeeper. If job manager failover works as expected, but the task
managers don't connect to the new job manager, I would suspect that
the task manager configuration is out of sync. Could you check this
please?

Moreover, it will be helpful to have a look at the jobmanager and task
manager logs to further investigate this. Can you share these?
(Privately works as well of course.)

– Ufuk




On Sat, Feb 27, 2016 at 1:28 AM, Welly Tambunan <if05041@gmail.com> wrote:
> typos
>
>
> We have tried this one the job manager can failover, but the task manager
> CAN'T be relocated to the new task manager. Is there some settings for this
> one ? Or is the task manager also can be relocate to the new job manager ?
>
> Cheers
>
> On Sat, Feb 27, 2016 at 7:27 AM, Welly Tambunan <if05041@gmail.com> wrote:
>>
>> Hi All,
>>
>> We have already try to setup the Job Manager HA based on the documentation
>> and using script and provided zookeeper. It works.
>>
>> However currently everything is done using start-cluster script that I
>> believe will require passwordlress ssh between node. We are restricted with
>> our environment so this one is not possible.
>>
>> Is it possible to setup the Job Manager HA manually ? By starting each job
>> manager with in each node and task manager. We have our zookeeper and hdfs
>> cluster already.
>>
>> We have tried this one the job manager can failover, but the task manager
>> can be relocated to the new task manager. Is there some settings for this
>> one ? Or is the task manager also can be relocate to the new job manager ?
>>
>> Any more details on the mechanism used on Job Manager HA and interaction
>> with Zookeeper ?
>>
>> Is task manager also registered on Zookeeper ? How they find the right job
>> manager master ?
>>
>>
>> Thanks for your help.
>>
>> Cheers
>> --
>> Welly Tambunan
>> Triplelands
>>
>> http://weltam.wordpress.com
>> http://www.triplelands.com
>
>
>
>
> --
> Welly Tambunan
> Triplelands
>
> http://weltam.wordpress.com
> http://www.triplelands.com

Mime
View raw message