From users-return-30500-archive-asf-public=cust-asf.ponee.io@cloudstack.apache.org Wed May 2 10:27:24 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 37D6618065D for ; Wed, 2 May 2018 10:27:23 +0200 (CEST) Received: (qmail 34275 invoked by uid 500); 2 May 2018 08:27:21 -0000 Mailing-List: contact users-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@cloudstack.apache.org Delivered-To: mailing list users@cloudstack.apache.org Received: (qmail 34250 invoked by uid 99); 2 May 2018 08:27:20 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 May 2018 08:27:20 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 194F3C0147 for ; Wed, 2 May 2018 08:27:20 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.001 X-Spam-Level: X-Spam-Status: No, score=-0.001 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=dimsi1.onmicrosoft.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id Ln3avUT03QfJ for ; Wed, 2 May 2018 08:27:17 +0000 (UTC) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0108.outbound.protection.outlook.com [104.47.1.108]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 827025F124 for ; Wed, 2 May 2018 08:27:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=DIMSI1.onmicrosoft.com; s=selector1-dimsi-fr; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=OLxm8Y82D2dGyDyQfs+PUScmUvPky1FgLIvJbCbDJCk=; b=floq7vKRW4Khvjge4EFcvWWC0Onku0wEPoMRuyKvIaDgzDg6g+DNAyU0oRRM3/U1oB0o11U7dMO46aOjNJ1goc/iZkDOzlHtHmW4ZTdUx4ClPEv6Zmkf5l5aTry0F1tpOmd+q7dVUnMQYHdPv5DFd2FDhB2D90eucrNi/5nZuSM= Received: from VI1PR0701MB1869.eurprd07.prod.outlook.com (10.167.197.17) by VI1PR0701MB2430.eurprd07.prod.outlook.com (10.168.138.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.735.6; Wed, 2 May 2018 08:27:07 +0000 Received: from VI1PR0701MB1869.eurprd07.prod.outlook.com ([fe80::4959:d1a1:c8a4:c414]) by VI1PR0701MB1869.eurprd07.prod.outlook.com ([fe80::4959:d1a1:c8a4:c414%7]) with mapi id 15.20.0735.006; Wed, 2 May 2018 08:27:07 +0000 From: Nicolas Bouige To: "users@cloudstack.apache.org" Subject: RE: certificate issue second mgmt-server Thread-Topic: certificate issue second mgmt-server Thread-Index: AQHT4GO4BUWJzNuk/k+vYfTRcvtXHaQas3r6gAFpF9A= Date: Wed, 2 May 2018 08:27:07 +0000 Message-ID: References: In-Reply-To: Accept-Language: fr-FR, en-US Content-Language: fr-FR X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=n.bouige@dimsi.fr; x-originating-ip: [80.11.25.232] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;VI1PR0701MB2430;7:VW4Y0k3l6g0CK9XFQa0V/66IAB47lDiikBQbxcnvkyrSwSZmFU4LBzQxFWvS9635FREhlS0+JRmrHyv02zoXZR9nf0SeTV+7a5Fdb91/PtrmouSu96x+vH7KdyqADKTCFfaFSL6/wUUQUP2234x9tVJYv/WXaydvr7FnNP8uoVXSrUuTv4vO4xE+uKlJYeuZr7GPS3qnxLm0Nn1sVI/JL48sxqxpWsrBvM33bISbWZu7g/lT6fY0vNEuMsl3yDCh x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(4652020)(7021125)(5600026)(4534165)(7022125)(4603075)(7168020)(4627221)(201702281549075)(7048125)(7024125)(7027125)(7028125)(7023125)(2017052603328)(7153060)(7193020);SRVR:VI1PR0701MB2430; x-ms-traffictypediagnostic: VI1PR0701MB2430: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(21532816269658); x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(6040522)(2401047)(5005006)(8121501046)(10201501046)(3231254)(944501410)(52105095)(93006095)(93001095)(3002001)(6041310)(20161123562045)(2016111802025)(20161123560045)(20161123564045)(20161123558120)(6043046)(6072148)(201708071742011);SRVR:VI1PR0701MB2430;BCL:0;PCL:0;RULEID:;SRVR:VI1PR0701MB2430; x-forefront-prvs: 06607E485E x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(39830400003)(346002)(366004)(396003)(39380400002)(376002)(55784002)(53754006)(377424004)(199004)(189003)(2906002)(3660700001)(6436002)(186003)(7736002)(66066001)(6916009)(14454004)(476003)(316002)(229853002)(11346002)(86362001)(551544002)(8936002)(2351001)(99286004)(74482002)(15974865002)(81166006)(5250100002)(446003)(3280700002)(2501003)(7696005)(25786009)(2900100001)(106356001)(478600001)(105586002)(26005)(76176011)(3846002)(81156014)(486006)(9686003)(59450400001)(68736007)(53936002)(55236004)(5660300001)(74316002)(305945005)(8676002)(33656002)(6116002)(6506007)(53546011)(55016002)(6246003)(97736004)(102836004)(6306002)(5640700003)(1730700003);DIR:OUT;SFP:1102;SCL:1;SRVR:VI1PR0701MB2430;H:VI1PR0701MB1869.eurprd07.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: dimsi.fr does not designate permitted sender hosts) x-microsoft-antispam-message-info: Es1KAfD7UcihjimrcJXpiUmXmzvj//C5cQwaEQabZUf3JZ0AxHktAoSb5MmbeiwmNd8fDUaVQD3x9rCz8IJwveo5GJelzLjme4kx2poJxxcu9O3qetnuVCEbhtDEnvzhAcmOpqqwcur5+nt8Q7VR/NQtTw74jXw1cG1sW58olCpv+D674v7OiqIVibEP+1TQMS+cffu48fSQgH/SlJon9g== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 4f7e1a07-29d1-4294-3327-08d5b0067e0e X-OriginatorOrg: dimsi.fr X-MS-Exchange-CrossTenant-Network-Message-Id: 4f7e1a07-29d1-4294-3327-08d5b0067e0e X-MS-Exchange-CrossTenant-originalarrivaltime: 02 May 2018 08:27:07.3164 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: bab0ba86-ddf4-4ac4-b09f-d48f7eb9d905 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0701MB2430 Hi Rohit, Thanks for your answer, i can't remember if i added the second node before = the end of the initialization maybe i was too impatient :/ I'll give a try this week with your workaround. Your workaround will affect also KVM server and System-VM ? So, i guess, it's not enought to delete the second node and redeploy it ? Best regards, N.B -----Message d'origine----- De=A0: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]=20 Envoy=E9=A0: mardi 1 mai 2018 12:56 =C0=A0: users@cloudstack.apache.org Objet=A0: Re: certificate issue second mgmt-server Hi Nicolas, Did you deploy multiple managements at the same time? When you deploy multi= ple management server(s), wait for the first management server to initializ= e database where it sets up some default offerings, global settings and the= root CA keypair and certificate. Only when you see the first management se= rver's UI in browser, proceed with deployment of other management server(s)= . For your environment, you can test this workaround and let me know if that = works for you: 1. Shutdown all the management server(s). 2. Delete ca keypair and cert: delete from configuration where name like "ca.plugin.root.private.key"; d= elete from configuration where name like "ca.plugin.root.public.key"; dele= te from configuration where name=3D"ca.plugin.root.ca.certificate"; 3. Start one management server and wait for it to complete internal setu= p, until you see the UI. 4. Start all the other management server(s). - Rohit ________________________________ From: Nicolas Bouige Sent: Monday, April 30, 2018 2:59:29 PM To: users@cloudstack.apache.org Subject: certificate issue second mgmt-server Hello All, I have an issue with one of my Cloudstack mgmt-server (4.11) The second node has been deployed with the command "cloudstack-setup-datab= ases cloud:dbpassword@dbhost" i didnt have any problem during few days and now sometimes i got an error o= n web GUI when i perfom some basic task, the error is "Resource [Host:1] is= unreachable: Host 1: Unable to reach the peer that the agent is connected" After a quick investigation, i had to stop cloudstack-management service fr= om second mgmt-server and i noticed a lot of messages related with ca-certi= ficate used by cloudstack : 2018-04-27 11:18:24,076 ERROR [c.c.u.n.Link] (StatsCollector-1:ctx-82335701= ) (logid:95fda6d7) SSL error caught during unwrap data: Received fatal aler= t: certificate_unknown, for local address=3D/172.16.22.61:60128, remote add= ress=3D/172.16.22.60:8250. The client may have invalid ca-certificates. 2018-04-27 11:18:24,076 WARN [c.c.a.m.ClusteredAgentManagerImpl] (StatsCol= lector-1:ctx-82335701) (logid:95fda6d7) Unable to connect to peer managemen= t server: 130719784044197, ip: 172.16.22.60 due to SSL: Fail to init SSL! j= ava.io.IOException: SSL: Handshake failed with peer management server '1307= 19784044197' on 172.16.22.60:8250 java.io.IOException: SSL: Fail to init SSL! java.io.IOException: SSL: Hands= hake failed with peer management server '130719784044197' on 172.16.22.60:8= 250 at com.cloud.agent.manager.ClusteredAgentManagerImpl.connectToPeer(= ClusteredAgentManagerImpl.java:529) at com.cloud.agent.manager.ClusteredAgentAttache.send(ClusteredAgen= tAttache.java:177) at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:398) at com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.j= ava:456) at com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.j= ava:362) at com.cloud.agent.manager.AgentManagerImpl.easySend(AgentManagerIm= pl.java:954) at com.cloud.resource.ResourceManagerImpl.getHostStatistics(Resourc= eManagerImpl.java:2645) at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod= AccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingRef= lection(AopUtils.java:338) at org.springframework.aop.framework.ReflectiveMethodInvocation.inv= okeJoinpoint(ReflectiveMethodInvocation.java:197) at org.springframework.aop.framework.ReflectiveMethodInvocation.pro= ceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.= invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.pro= ceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkD= ynamicAopProxy.java:212) at com.sun.proxy.$Proxy178.getHostStatistics(Unknown Source) at com.cloud.server.StatsCollector$HostCollector.runInContext(Stats= Collector.java:438) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.r= un(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext= $1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext= .callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext= .runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run= (ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.ja= va:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFuture= Task.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFuture= Task.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor.java:624) at java.lang.Thread.run(Thread.java:748) 2018-04-27 11:18:24,077 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollect= or-1:ctx-82335701) (logid:95fda6d7) Seq 9-9075597674081682614: Unable to fo= rward null 2018-04-27 11:18:24,177 ERROR [c.c.u.n.Link] (StatsCollector-1:ctx-82335701= ) (logid:95fda6d7) SSL error caught during unwrap data: Received fatal aler= t: certificate_unknown, for local address=3D/172.16.22.61:60130, remote add= ress=3D/172.16.22.60:8250. The client may have invalid ca-certificates. 2018-04-27 11:18:24,177 WARN [c.c.a.m.ClusteredAgentManagerImpl] (StatsCol= lector-1:ctx-82335701) (logid:95fda6d7) Unable to connect to peer managemen= t server: 130719784044197, ip: 172.16.22.60 due to SSL: Fail to init SSL! j= ava.io.IOException: SSL: Handshake failed with peer management server '1307= 19784044197' on 172.16.22.60:8250 Im not familiar with the using of self-signed certificate in cloudstack, do= you know where i can find out more information to investigate deeper ? or = if you have any idea ? I tried to check keystore on both mgmt-server but i need a password i havnt= ... Thanks upfront, Have a nice day, Best regards, Nicolas Bouige DIMSI cloud.dimsi.fr 4, avenue Laurent Cely Tour d'Asni=E8re - 92600 Asni=E8re sur Seine T/ +33 (0)6 28 98 53 40 rohit.yadav@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue =20 =20