cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristian Liivak <k...@wavecom.ee>
Subject Re: System VM Issues 4.11?
Date Tue, 20 Feb 2018 21:25:42 GMT
Hi Rohit,

Made another shot. 

Recreated all systemvm before update and get rid most of errors had before.

Like before no host nor systemvm was present in gui
However i saw correct numbers under infrastructure.
I attached some pics form gui  host and systemvm  selection.

I did destroyd old systemvms last time 
New ones started without network. 
New templates were present and new systemvm are created from 4.11 templates, checked from
xen console.

My management server resides in different vmware enviroment, however it have same management
network like all my xenservers.

I started with 4.9, where dynamic roles  were allready introduced, but i doublechecked.

My old enviroment was configured with different port using custom ssl, so therefore no cache.

Enviroment is only xenserver7.0 hosts 

Uploaded new management logs and also /var/log/messages where update logs are present https://wavecom.ee/files/logs.rar

Maybe you can find time to check..

And i can allways rollback management where i was to have additional info 

PS what you mean about : reset host flags and reconnect on all such hosts. i tried add hosts
agan in gui bot that ended with failure.

Lugupidamisega / Regards
 
Kristian Liivak

Tegevjuht / Executive director

WaveCom As
Endla 16, 10142 Tallinn
Estonia
Tel: +3726850001
Gsm: +37256850001
E-mail: kris@wavecom.ee
Skype: kristian.liivak
http://www.wavecom.ee
http://www.facebook.com/wavecom.ee

----- Original Message -----
From: "rohit yadav" <rohit.yadav@shapeblue.com>
To: "users" <users@cloudstack.apache.org>
Sent: Sunday, February 18, 2018 3:52:00 PM
Subject: Re: System VM Issues 4.11?

Thanks Kristian,


Please confirm:

- Did you see the new 4.11 systemvmtemplate and tried to destroy old systemvms and/or virtual
routers?

- Are all the xenserver hosts version 7.0 in all your clusters?

- Was the upgrade in-place (on the same host), or on a different server?

- Is dynamic roles enabled or you're still using the old/deprecated static roles i.e. commands.properties
file?

- Did you try accessing the UI in incognito mode or after clearing the browser cache?


Looking at the logs, some errors:


>  2018-01-01 00:01:22,481 ERROR [c.c.u.s.SshHelper] (DirectAgent-29:ctx-62676ad7) (logid:48c55dee)
SSH execution of command /opt/cloud/bin/router_proxy.sh netusage.sh 169.254.3.20 -g has an
error status code in return. Result output: /root/func.sh: line 61: /tmp/1514746614947437999-28844-biglock.lock:
Read-only file system

> 2018-01-01 00:01:22,482 WARN  [c.c.h.x.r.w.x.XenServer56NetworkUsageCommandWrapper] (DirectAgent-29:ctx-62676ad7)
(logid:48c55dee) Failed to get network usage


This looks like a case where an old (pre 4.11) or new (4.11) based virtual router ended up
having read-only file system. If there are old virtual routers they need to be removed, restart
network with cleanup=true.


> 2018-01-01 00:01:24,503 DEBUG [c.c.h.x.r.XenServerConnectionPool] (ClusteredAgentManager
Timer:ctx-198e0371) (logid:509434bb) Unable to create master connection to host(172.16.10.13)
, due to org.apache.xmlrpc.XmlRpcException: Failed to read server's response: connect timed
out                          > 2018-01-01 00:01:24,503 DEBUG [c.c.h.Status] (ClusteredAgentManager
Timer:ctx-198e0371) (logid:509434bb) Transition:[Resource state = Enabled, Agent event = AgentDisconnected,
Host id = 864, name = xen-13-10-b1-khi02]

I'm not sure why your mgmt server was unable to make a connection with existing xenserver
host. After a major version upgrade, on reconnection scripts will be copied to your xenserver
host. In case that failed (because I don't have your upgrade logs), you'll need to reset host
flags and reconnect on all such hosts.

> 2018-01-01 00:01:26,367 ERROR [c.c.u.s.SshHelper] (AgentTaskPool-12:ctx-cfea21e4) (logid:c2b516fe)
SSH execution of command xe sm-list | grep "resigning of duplicates" has an error status code
in return. Result output:


Not sure about these ones.


> 2018-01-01 00:40:09,411 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-271:ctx-c9e202e1)
(logid:1f6aafe2) Unable to start i-1744-15758-VM due to-

com.cloud.utils.exception.CloudRuntimeException: createIsoSRbyURI failed! mountpoint: 172.16.2.37/nfs/KHI03/template/tmpl/1326/465
due to SR_BACKEND_FAILURE_222Could not mount the directory specified in Device Configuration
>---at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.createIsoSRbyURI(CitrixResourceBase.java:988)

Can you check if your storage pools are accessible to your hosts?


Lastly, if you attempt upgrade again (or perhaps on a test environment), do keep the logs
for future reference for example sharing with us in case it fails for you again.



- Rohit

<https://cloudstack.apache.org>



________________________________
From: Kristian Liivak <kris@wavecom.ee>
Sent: Saturday, February 17, 2018 8:49:26 PM
To: users
Subject: Re: System VM Issues 4.11?

Hi Rohit

Here it is https://wavecom.ee/files/management-server.rar
Log beginning little bit earlier then upgrade.
I did,nt fallow errors during gui access this time.
If it helps i can allways revert management server where i was in that moment and grab more
info.

Lugupidamisega / Regards

Kristian Liivak

WaveCom As
Endla 16, 10142 Tallinn
Estonia
Tel: +3726850001
Gsm: +37256850001
E-mail: kris@wavecom.ee
Skype: kristian.liivak
http://www.wavecom.ee
http://www.facebook.com/wavecom.ee


rohit.yadav@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

----- Original Message -----
From: "rohit yadav" <rohit.yadav@shapeblue.com>
To: "users" <users@cloudstack.apache.org>
Sent: Saturday, February 17, 2018 9:15:56 PM
Subject: Re: System VM Issues 4.11?

Hi Kristian,


Yes, can you share the logs and if you saw any sql/db or other errors when you tried to access
the UI?


- Rohit

<https://cloudstack.apache.org>



________________________________
From: Kristian Liivak <kris@wavecom.ee>
Sent: Saturday, February 17, 2018 3:07:04 PM
To: users
Subject: Re: System VM Issues 4.11?

Hi All

I also upgraded from 4.10->4.11 and had similar issues ,even more.
After upgrade, Gui under infrastructure i didnt show any host nor systemvm anymore , except
VR
Destroyd consoleproxy, new was created but no network running.

Enviroment is Xenserver7 and i also runned before upgrade sql update mentioned below .
Before reverting back i grabbed all log files. If anyone is intrested i can make them downloadable.


Lugupidamisega / Regards

Kristian Liivak

WaveCom As
Endla 16, 10142 Tallinn
Estonia
Tel: +3726850001
Gsm: +37256850001
E-mail: kris@wavecom.ee
Skype: kristian.liivak
http://www.wavecom.ee
http://www.facebook.com/wavecom.ee


rohit.yadav@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue



----- Original Message -----
From: "Paul Angus" <paul.angus@shapeblue.com>
To: "users" <users@cloudstack.apache.org>
Sent: Friday, February 16, 2018 10:45:26 AM
Subject: RE: System VM Issues 4.11?

Sami,

Have you run the update step which is mentioned here?
http://markmail.org/message/f42kqr3mx4r4hgih

I don't KNOW if it is causing your problem, but if not, it won't help if it is missing.


Kind regards,

Paul Angus

paul.angus@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue




-----Original Message-----
From: sami [mailto:sami.rajala@fapps.com]
Sent: 16 February 2018 07:44
To: users@cloudstack.apache.org
Subject: Fwd: System VM Issues 4.11?


Hi,

Actually I have a related issue, The System VMs stay in starting state.
when I look situation on XenCenter, I see CPVM /SSVM running, but does not deploy configuration
(no linklocal/private/publi IP on those VM¹s)

I did not try VPC router yet, The Network routers starting ok.


ACS upgraded 4.10 to 4.11 on Centos 6
xenserver 7.0
Advance network

Regards
-Sami


> 15.2.2018 20.14, "Rohit Yadav" <rohit.yadav@shapeblue.com>:
>
>> Hi Benjamin,
>>
>> Starting 4.11, systemvms don't reboot after the patching operation so
>> the shared observation matches expectations. Do you see vpc router
>> vms stuck in starting state, can you share the logs and details of
>> your hypervisor and other environment details?
>>
>>
>> - Rohit
>>
>> <https://cloudstack.apache.org>
>>
>>
>>
>> ________________________________
>> From: Benjamin Naber <benjamin.naber@coders-area.de>
>> Sent: Thursday, February 15, 2018 6:01:51 PM
>> To: users
>> Subject: System VM Issues 4.11?
>>
>> Hi @all,
>>
>> I've installed 4.11 from shape blue CentOS 7 Repository.
>> System VM and Router VM Upgrades working well. But have some issues
>> if I deploy a new Vpc. Router VMs stalled in state starting.
>> Inspecting the hypervisor shows that the router VM doesn't reboot and
>> doesn't deploy the configuration. Anyone other have the same issues?
>>
>> Kind Regards
>>
>> Benjamin
>>
>> Von meinem Huawei-Mobiltelefon gesendet
>>
>> rohit.yadav@shapeblue.com
>> www.shapeblue.com<http://www.shapeblue.com>
>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>>
>>
Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message