From user-return-23470-archive-asf-public=cust-asf.ponee.io@flink.apache.org Sat Oct 6 14:35:36 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4752118061A for ; Sat, 6 Oct 2018 14:35:35 +0200 (CEST) Received: (qmail 79902 invoked by uid 500); 6 Oct 2018 12:35:34 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 79891 invoked by uid 99); 6 Oct 2018 12:35:34 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 06 Oct 2018 12:35:34 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 8915C1A0FAA for ; Sat, 6 Oct 2018 12:35:33 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.889 X-Spam-Level: * X-Spam-Status: No, score=1.889 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id M5UR9uL2_XBT for ; Sat, 6 Oct 2018 12:35:32 +0000 (UTC) Received: from mail-lj1-f195.google.com (mail-lj1-f195.google.com [209.85.208.195]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 62D175F217 for ; Sat, 6 Oct 2018 12:35:31 +0000 (UTC) Received: by mail-lj1-f195.google.com with SMTP id u21-v6so13928617lja.8 for ; Sat, 06 Oct 2018 05:35:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=16hVKEp36p3MD81lTT7bKZJDYqqRrcZsbM5Pndv+ESQ=; b=hcOWpW/qorKQtLQ/x2XfsUpjVFOGFcgy3/8X+w8OkDaK8PaRoOA77nV1aZYynZARsC fKdp8fex6RG7gvaRZFSJUy4tVjQWZq96MhDfRr0JDvU5wN+nCxgWmCsspxeoEVzK6a54 JmZYjK9T69J6pvp8rebCC3w7KPpeEy/9+7fjhvfuiWE+NM/9fhC4qJzCi0++Sbewsutc d2DdNN7RVWpdTjb5zA5M+REfJDEK66gN89x4+tu7oXPgYDA7X0uZltwaofXSvgh+fjKJ wbE8lTN/n3BdmQMbY3erIcMObBLBJdHe4Nu6u/8rKzILdKGRCopHp6wueB/cnyEvId00 e+EA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=16hVKEp36p3MD81lTT7bKZJDYqqRrcZsbM5Pndv+ESQ=; b=XBcmUWsSCtHQQjVf+zUERXfB1ZV3tIELn6MV/XP/4lkgHyiC5Qehuo20jTl/YlPbOb vP9ZQtlJ11zvPZhgEmDyC1bMlsk0SYnEcrQEcBASRilMHybVNxZ1jTYkHNyjfMO4iXCV p5B+WLmfcp22wiPK2YfTBmSnnHE/Hl9nRf6wuzu2LOKtVI+NIxX39zroIvYZ0tr7qw/W Q/LLzt6wCwiW4pTDd/puQs/NY6Kf6S1gOUl9wbu6gZ65fPpb06Whpw6FBep+xup3SRzX eY1JDXVaoi6lXeI1rasfVT3ooubZs69smMNFbZhmdpcA8A8VouR2pRxLZCl+beAoRPir zMeQ== X-Gm-Message-State: ABuFfogtMlIWWdOU+APssGxYktg/PmxyNF/i5WoFovFn9e3HJEp/cIwc x55ZRM63TwerP6qb/0ZSwG8WXov2eqo9jTmU+4PVKg== X-Google-Smtp-Source: ACcGV62SKO9/YK1VLz12zrkkI5SYKvt2dvu5ADVkscecyEVHNMtBqvoH1bnGbjC5PrOR1RCxVmWpFNqbZRvi8NRwMqM= X-Received: by 2002:a2e:20f:: with SMTP id 15-v6mr9833222ljc.141.1538829330620; Sat, 06 Oct 2018 05:35:30 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Vinay Patil Date: Sat, 6 Oct 2018 18:05:18 +0530 Message-ID: Subject: Re: Unable to start session cluster using Docker To: Till Rohrmann Cc: user Content-Type: multipart/alternative; boundary="0000000000009f24a805778e9f46" --0000000000009f24a805778e9f46 Content-Type: text/plain; charset="UTF-8" Thank you Till, I am able to start the session-cluster now. Regards, Vinay Patil On Fri, Oct 5, 2018 at 8:15 PM Till Rohrmann wrote: > Hi Vinay, > > are you referring to flink-contrib/docker-flink/docker-compose.yml? We > recently fixed the command line parsing with Flink 1.5.4 and 1.6.1. Due to > this, the removal of the second command line parameter intended to be > introduced with 1.5.0 and 1.6.0 (see > https://issues.apache.org/jira/browse/FLINK-8696) became visible. The > docker-compose.yml file has not yet been updated. I will do this right away > and push the changes to the 1.5, 1.6 and master branch. Sorry for the > inconveniences. As a local fix for you, please go to > flink-contrib/docker-flink/docker-entrypoint.sh:33 and remove the cluster > parameter from this line. > > Cheers, > Till > > On Thu, Oct 4, 2018 at 8:30 PM Vinay Patil > wrote: > >> Hi, >> >> I have used the docker-compose file for creating the cluster as shown in >> the documentation. The web ui is started successfully, however, the task >> managers are unable to join. >> >> Job Manager container logs: >> >> 018-10-04 18:13:13,907 INFO >> org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Rest >> endpoint listening at cluster:8081 >> >> 2018-10-04 18:13:13,907 INFO >> org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - >> http://cluster:8081 was granted leadership with >> leaderSessionID=00000000-0000-0000-0000-000000000000 >> >> 2018-10-04 18:13:13,907 INFO >> org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Web >> frontend listening at http://cluster:8081 >> >> 2018-10-04 18:13:14,012 INFO >> org.apache.flink.runtime.resourcemanager.StandaloneResourceManager - >> ResourceManager akka.tcp://flink@cluster:6123/user/resourcemanager was >> granted leadership with fencing token 00000000000000000000000000000000 >> >> 2018-10-04 18:13:14,013 INFO >> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager - >> Starting the SlotManager. >> >> 2018-10-04 18:13:14,026 INFO >> org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Dispatcher >> akka.tcp://flink@cluster:6123/user/dispatcher was granted leadership >> with fencing token 00000000-0000-0000-0000-000000000000 >> >> Not sure why it says Web Frontend listening at cluster:8081 when the job >> manager rpc address is specified to jobmanager >> >> Task Manager Container Logs: >> >> 018-10-04 18:19:18,818 INFO >> org.apache.flink.runtime.taskexecutor.TaskExecutor - Connecting >> to ResourceManager akka.tcp://flink@jobmanager >> :6123/user/resourcemanager(00000000000000000000000000000000). >> >> 2018-10-04 18:19:18,818 INFO >> org.apache.flink.runtime.filecache.FileCache - User file >> cache uses directory >> /tmp/flink-dist-cache-1bd95c51-3031-42ab-b782-14a0023921e5 >> >> 2018-10-04 18:19:28,850 INFO >> org.apache.flink.runtime.taskexecutor.TaskExecutor - Could not >> resolve ResourceManager address akka.tcp://flink@jobmanager:6123/user/resourcemanager, >> retrying in 10000 ms: Ask timed out on >> [ActorSelection[Anchor(akka.tcp://flink@jobmanager:6123/), >> Path(/user/resourcemanager)]] after [10000 ms]. Sender[null] sent message >> of type "akka.actor.Identify". >> >> >> I have even tried to set JOB_MANAGER_RPC_ADDRESS=cluster in in >> docker-compose file, it does not work. >> Even "cluster" and "jobmanager" points to localhost in /etc/hosts file. >> >> Can you please let me know what is the issue here. >> >> Regards, >> Vinay Patil >> > --0000000000009f24a805778e9f46 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Thank you Till, I am able to start the sess= ion-cluster now.=C2=A0

<= div>
Regards,
Vinay Patil


On Fri, Oct 5, 201= 8 at 8:15 PM Till Rohrmann <troh= rmann@apache.org> wrote:
Hi Vinay,

are you referri= ng to flink-contrib/docker-flink/docker-compose.yml? We recently fixed the = command line parsing with Flink 1.5.4 and 1.6.1. Due to this, the removal o= f the second command line parameter intended to be introduced with 1.5.0 an= d 1.6.0 (see https://issues.apache.org/jira/browse/FLINK-8696) beca= me visible. The docker-compose.yml file has not yet been updated. I will do= this right away and push the changes to the 1.5, 1.6 and master branch. So= rry for the inconveniences. As a local fix for you, please go to flink-cont= rib/docker-flink/docker-entrypoint.sh:33 and remove the cluster parameter f= rom this line.

Cheers,
Till
<= /div>
On Thu, Oct 4, 2018 at= 8:30 PM Vinay Patil <vinay18.patil@gmail.com> wrote:
Hi,

I have used the docker-com= pose file for creating the cluster as shown in the documentation. The web u= i=C2=A0is started successfully, however, the task managers are unable to jo= in.

Job Manager container logs:

018-10-04 18:13:13= ,907 INFO=C2=A0 org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint= =C2=A0 =C2=A0 - Rest endpoint listening at cluster:8081

2018-10-04= 18:13:13,907 INFO=C2=A0 org.apache.flink.runtime.dispatcher.DispatcherRest= Endpoint=C2=A0 =C2=A0 - h= ttp://cluster:8081 was granted leadership with leaderSessionID=3D000000= 00-0000-0000-0000-000000000000

2018-10-04= 18:13:13,907 INFO=C2=A0 org.apache.flink.runtime.dispatcher.DispatcherRest= Endpoint=C2=A0 =C2=A0 - Web frontend listening at http://cluster:8081

2018-10-04 18:1= 3:14,012 INFO=C2=A0 org.apache.flink.runtime.resourcemanager.StandaloneReso= urceManager=C2=A0 - ResourceManager akka.tcp://flink@cluster:6123/user/reso= urcemanager was granted leadership with fencing token 000000000000000000000= 00000000000

2018-10-04= 18:13:14,013 INFO=C2=A0 org.apache.flink.runtime.resourcemanager.slotmanag= er.SlotManager=C2=A0 - Starting the SlotManager.

2018-10-04= 18:13:14,026 INFO=C2=A0 org.apache.flink.runtime.dispatcher.StandaloneDisp= atcher=C2=A0 =C2=A0 =C2=A0 - Dispatcher akka.tcp://flink@cluster:6123/user/= dispatcher was granted leadership with fencing token 00000000-0000-0000-000= 0-000000000000


Not sure why it says Web= Frontend listening at=C2=A0cluster:8081 when the job manager rpc address i= s specified to jobmanager

Task Manager Container L= ogs:

018-10-04 18:19:18,818 INFO=C2=A0 org.apache.flink.runt= ime.taskexecutor.TaskExecutor=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - Co= nnecting to ResourceManager akka.tcp://flink@jobmanager:6123/user/resourcem= anager(00000000000000000000000000000000).

2018-10-04= 18:19:18,818 INFO=C2=A0 org.apache.flink.runtime.filecache.FileCache=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - User file cache = uses directory /tmp/flink-dist-cache-1bd95c51-3031-42ab-b782-14a0023921e5

2018-10-04= 18:19:28,850 INFO=C2=A0 org.apache.flink.runtime.taskexecutor.TaskExecutor= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - Could not resolve ResourceManag= er address akka.tcp://flink@jobmanager:6123/user/resourcemanager, retrying = in 10000 ms: Ask timed out on [ActorSelection[Anchor(akka.tcp://flink@jobma= nager:6123/), Path(/user/resourcemanager)]] after [10000 ms]. Sender[null] = sent message of type "akka.actor.Identify".

=

I have even tried to set=C2=A0JOB_MANAGER_RPC_ADDRESS=3Dcluster in =C2=A0 in do= cker-compose file, it does not work.
Even "cluster" and= "jobmanager" points to localhost in /etc/hosts file.
<= br>
Can you please let me know what is the issue here.
=
Regards,
Vinay Patil
--0000000000009f24a805778e9f46--