cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremy Peterson <jpeter...@acentek.net>
Subject RE: Recreating SystemVM's
Date Tue, 20 Jun 2017 17:33:20 GMT
Ok so my issues have not gone away.

I have two hosts that have not rebooted yet tonight I will be maintenancing those hosts out
and migrating vm's away from those hosts and then performing a reboot of the host and installing
a couple xenserver updates.  

One thing is I am not getting the CANNOT ATTACH NETWORK error anymore which is cool but.

https://drive.google.com/open?id=0B5IXhrpPAT9qQ0FFUmRyRjN4NlE

Take a look at creation of VM 20685

2017-06-20 12:15:48,083 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-82:ctx-c39fa1f8
job-1042/job-138603 ctx-c17ce6fc) Found a potential host id: 1 name: Flex-Xen2.flexhost.local
and associated storage pools for this VM
2017-06-20 12:15:48,084 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-82:ctx-c39fa1f8
job-1042/job-138603 ctx-c17ce6fc) Returning Deployment Destination: Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(25604|ROOT-->Pool(5))]
2017-06-20 12:15:48,084 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-82:ctx-c39fa1f8
job-1042/job-138603 ctx-c17ce6fc) Deployment found  - P0=VM[SecondaryStorageVm|s-20685-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage(Volume(25604|ROOT-->Pool(5))]

So it found a host and storage pool 

Networks were already created on line 482-484

But then look it fails on create volume  UUID is invalid???


2017-06-20 12:15:48,262 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-88:ctx-c51dafa0
job-342/job-138604 ctx-75edebb0) VM is being created in podId: 1
2017-06-20 12:15:48,264 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-88:ctx-c51dafa0
job-342/job-138604 ctx-75edebb0) Network id=200 is already implemented
2017-06-20 12:15:48,269 DEBUG [c.c.n.g.PodBasedNetworkGuru] (Work-Job-Executor-82:ctx-c39fa1f8
job-1042/job-138603 ctx-c17ce6fc) Allocated a nic NicProfile[81905-20685-0493941d-d193-4325-84bc-d325a8900332-10.90.2.207-null
for VM[SecondaryStorageVm|s-20685-VM]
2017-06-20 12:15:48,280 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-82:ctx-c39fa1f8
job-1042/job-138603 ctx-c17ce6fc) Network id=203 is already implemented
2017-06-20 12:15:48,290 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-88:ctx-c51dafa0
job-342/job-138604 ctx-75edebb0) Network id=202 is already implemented
2017-06-20 12:15:48,316 DEBUG [c.c.n.g.StorageNetworkGuru] (Work-Job-Executor-82:ctx-c39fa1f8
job-1042/job-138603 ctx-c17ce6fc) Allocated a storage nic NicProfile[81906-20685-0493941d-d193-4325-84bc-d325a8900332-10.83.2.205-null
for VM[SecondaryStorageVm|s-20685-VM]
2017-06-20 12:15:48,336 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-82:ctx-c39fa1f8
job-1042/job-138603 ctx-c17ce6fc) Checking if we need to prepare 1 volumes for VM[SecondaryStorageVm|s-20685-VM]
2017-06-20 12:15:48,342 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl] (Work-Job-Executor-82:ctx-c39fa1f8
job-1042/job-138603 ctx-c17ce6fc) template 1 is already in store:5, type:Image
2017-06-20 12:15:48,344 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl] (Work-Job-Executor-82:ctx-c39fa1f8
job-1042/job-138603 ctx-c17ce6fc) template 1 is already in store:5, type:Primary
2017-06-20 12:15:48,346 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-88:ctx-c51dafa0
job-342/job-138604 ctx-75edebb0) Network id=201 is already implemented
2017-06-20 12:15:48,372 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl] (Work-Job-Executor-88:ctx-c51dafa0
job-342/job-138604 ctx-75edebb0) Releasing ip address for instance=49817
2017-06-20 12:15:48,381 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (Work-Job-Executor-82:ctx-c39fa1f8
job-1042/job-138603 ctx-c17ce6fc) copyAsync inspecting src type TEMPLATE copyAsync inspecting
dest type VOLUME
2017-06-20 12:15:48,386 DEBUG [c.c.a.t.Request] (Work-Job-Executor-82:ctx-c39fa1f8 job-1042/job-138603
ctx-c17ce6fc) Seq 16-3622864425242874354: Sending  { Cmd , MgmtId: 345050411715, via: 16(Flex-Xen6.flexhost.local),
Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"ab6f3bcd-4c3e-4a7a-9f8b-45a822dbaaaf","origUrl":"http://download.cloud.com/templates/4.5/systemvm64template-4.5-xen.vhd.bz2","uuid":"8a4039f2-bb71-11e4-8c76-0050569b1662","id":1,"format":"VHD","accountId":1,"checksum":"2b15ab4401c2d655264732d3fc600241","hvm":false,"displayText":"SystemVM
Template (XenServer)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN2-LUN0","id":5,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN2-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN2-LUN0/?ROLE=Primary&STOREUUID=FlexSAN2-LUN0"}},"name":"routing-1","hypervisorType":"XenServer"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"4dba9def-2657-430e-8cd8-9369aebcaa25","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN2-LUN0","id":5,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN2-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN2-LUN0/?ROLE=Primary&STOREUUID=FlexSAN2-LUN0"}},"name":"ROOT-20685","size":2689602048,"volumeId":25604,"vmName":"s-20685-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":25604,"deviceId":0,"hypervisorType":"XenServer"}},"executeInSequence":true,"options":{},"wait":0}}]
}
2017-06-20 12:15:48,386 DEBUG [c.c.a.t.Request] (Work-Job-Executor-82:ctx-c39fa1f8 job-1042/job-138603
ctx-c17ce6fc) Seq 16-3622864425242874354: Executing:  { Cmd , MgmtId: 345050411715, via: 16(Flex-Xen6.flexhost.local),
Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"ab6f3bcd-4c3e-4a7a-9f8b-45a822dbaaaf","origUrl":"http://download.cloud.com/templates/4.5/systemvm64template-4.5-xen.vhd.bz2","uuid":"8a4039f2-bb71-11e4-8c76-0050569b1662","id":1,"format":"VHD","accountId":1,"checksum":"2b15ab4401c2d655264732d3fc600241","hvm":false,"displayText":"SystemVM
Template (XenServer)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN2-LUN0","id":5,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN2-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN2-LUN0/?ROLE=Primary&STOREUUID=FlexSAN2-LUN0"}},"name":"routing-1","hypervisorType":"XenServer"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"4dba9def-2657-430e-8cd8-9369aebcaa25","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN2-LUN0","id":5,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN2-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN2-LUN0/?ROLE=Primary&STOREUUID=FlexSAN2-LUN0"}},"name":"ROOT-20685","size":2689602048,"volumeId":25604,"vmName":"s-20685-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":25604,"deviceId":0,"hypervisorType":"XenServer"}},"executeInSequence":true,"options":{},"wait":0}}]
}
2017-06-20 12:15:48,386 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-74:ctx-0acdd419) Seq
16-3622864425242874354: Executing request
2017-06-20 12:15:48,387 DEBUG [c.c.n.g.PodBasedNetworkGuru] (Work-Job-Executor-88:ctx-c51dafa0
job-342/job-138604 ctx-75edebb0) Allocated a nic NicProfile[49817-12662-629b85e7-ce19-4568-9df7-143c76d24300-10.90.2.204-null
for VM[ConsoleProxy|v-12662-VM]


So how do I check UUID's to validate that they are correct ? 

2017-06-20 12:15:48,391 DEBUG [c.c.h.x.r.XenServerStorageProcessor] (DirectAgent-74:ctx-0acdd419)
Catch Exception com.xensource.xenapi.Types$UuidInvalid :VDI getByUuid for uuid: ab6f3bcd-4c3e-4a7a-9f8b-45a822dbaaaf
failed due to The uuid you supplied was invalid.
2017-06-20 12:15:48,391 WARN  [c.c.h.x.r.XenServerStorageProcessor] (DirectAgent-74:ctx-0acdd419)
Unable to create volume; Pool=volumeTO[uuid=4dba9def-2657-430e-8cd8-9369aebcaa25|path=null|datastore=PrimaryDataStoreTO[uuid=FlexSAN2-LUN0|name=null|id=5|pooltype=PreSetup]];
Disk:
com.cloud.utils.exception.CloudRuntimeException: Catch Exception com.xensource.xenapi.Types$UuidInvalid
:VDI getByUuid for uuid: ab6f3bcd-4c3e-4a7a-9f8b-45a822dbaaaf failed due to The uuid you supplied
was invalid.




Jeremy

-----Original Message-----
From: Jeremy Peterson [mailto:jpeterson@acentek.net] 
Sent: Thursday, June 15, 2017 4:20 PM
To: users@cloudstack.apache.org; S. Brüseke - proIO GmbH <s.brueseke@proio.com>
Subject: RE: Recreating SystemVM's

What type of networking are you using on the XenServers?
	XenServers are connected with 6 nic's per host connected to separate nexus 5k switches
	NIC 0 and NIC 1 are Bond 0+1 10Gb nics
	NIC 2 and NIC 3 are Bond 2+3 10Gb nics
	NIC 4 and NIC 5 are Bond 4+5 2Gb nics 
	Cloudstack is running Advanced networking
	Bond 0+1 is primary storage
	Bond 2+3 is secondary storage 
	Bond 4+5 is Management
What version of os does the ms run on?
	CentOS release 6.9 (Final)
What are the systemvm templates defined in your env?
	http://cloudstack.apt-get.eu/systemvm/4.5/systemvm64template-4.5-xen.vhd.bz2
What is the version of the systemvm.iso?
	Successfully installed system VM template  to /secondary/template/tmpl/1/1/
	I just reinstalled systemvm's from the above 4.5-xen.vhd What is the capacity you have in
your (test) environment?
	This is a production enviroment and currently cloudstack shows the following.
	Public IP Addresses 61%
	VLAN 35%
	Management IP Addresses 20%
	Primary Storage 44%
	CPU 21%
	Memory 5%
	Of cource Secondary Storage shows 0%
What is the host os version for the hypervisors?
	XenServer 6.5 SP1
What is the management network range?
	management.network.cidr 10.90.1.0/24
What are the other physical networks?
	?? Not sure what more you need
What storage do you use?
	Primary - ISCSI
	Secondary - NFS
Is it reachable from the systemvm?
	All of my CS management servers have internet access Is the big bad internet reachable for
your SSVM’s public interface?
	My SSVM does not go online but yes the public network is the same as the VR public vlan and
all instances behind VR's are connected to the internet at this time

Jeremy


-----Original Message-----
From: Daan Hoogland [mailto:daan.hoogland@shapeblue.com]
Sent: Thursday, June 15, 2017 9:34 AM
To: users@cloudstack.apache.org; S. Brüseke - proIO GmbH <s.brueseke@proio.com>
Subject: Re: Recreating SystemVM's

Your problem might be like what Swen says, Jeremy but also a wrong systemvm offering or a
fault in your management network definition.
I am going to sum up some trivialities so bear with me;

What type of networking are you using on the XenServers?
What version of os does the ms run on?
What are the systemvm templates defined in your env?
What is the version of the systemvm.iso?
What is the capacity you have in your (test) environment?
What is the host os version for the hypervisors?
What is the management network range?
What are the other physical networks?
What storage do you use?
Is it reachable from the systemvm?
Is the big bad internet reachable for your SSVM’s public interface?

And of course,

How is the weather, where you are at?

I am not sure any of these question is going to lead you in the right direction but one of
them should.

On 15/06/17 13:56, "S. Brüseke - proIO GmbH" <s.brueseke@proio.com> wrote:

    I once did have some similar problem with my systemvms and my root cause was that in the
global settings it referred to the wrong systemvm template. I am not sure if this helps you,
but wanted to tell you.
    
    Mit freundlichen Grüßen / With kind regards,
    
    Swen
    
    -----Ursprüngliche Nachricht-----
    Von: Jeremy Peterson [mailto:jpeterson@acentek.net] 
    Gesendet: Donnerstag, 15. Juni 2017 01:55
    An: users@cloudstack.apache.org
    Betreff: RE: Recreating SystemVM's
    
    Hahaha.  The best response ever. 
    
    I dug through these emails and someone had soft of the same log messages cannot attach
network and blamed xenserver. Ok I'm cool with that but why oh why is it only system vms?
    
    Jeremy
    ________________________________________
    From: Imran Ahmed [imran@eaxiom.net]
    Sent: Wednesday, June 14, 2017 6:22 PM
    To: users@cloudstack.apache.org
    Subject: RE: Recreating SystemVM's
    
    Yes,
    
    -----Original Message-----
    From: Jeremy Peterson [mailto:jpeterson@acentek.net]
    Sent: Wednesday, June 14, 2017 9:59 PM
    To: users@cloudstack.apache.org
    Subject: RE: Recreating SystemVM's
    
    Is there anyone out there reading these messages?
    
    Am I just not seeing responses?
    
    Jeremy
    
    
    -----Original Message-----
    From: Jeremy Peterson [mailto:jpeterson@acentek.net]
    Sent: Wednesday, June 14, 2017 8:12 AM
    To: users@cloudstack.apache.org
    Subject: RE: Recreating SystemVM's
    
    I opened an issue since this is still an issue.  CLOUDSTACK-9960
    
    Jeremy
    
    -----Original Message-----
    From: Jeremy Peterson [mailto:jpeterson@acentek.net]
    Sent: Sunday, June 11, 2017 9:10 AM
    To: users@cloudstack.apache.org
    Subject: Re: Recreating SystemVM's
    
    Any other suggestions?
    
    I am going to be scheduling to run XenServer updates.  But this all points back to CANNOT_ATTACH_NETWORk.
    
    I've verified nothing is active on the Public IP space that those two VM's were living
on.
    
    Jeremy
    ________________________________________
    From: Jeremy Peterson <jpeterson@acentek.net>
    Sent: Friday, June 9, 2017 9:58 AM
    To: users@cloudstack.apache.org
    Subject: RE: Recreating SystemVM's
    
    I see the vm's try to create on a host that I just removed from maintenance mode to install
updates and here are the logs
    
    I don't see anything that sticks out to me as a failure message.
    
    Jun  9 09:53:54 Xen3 SM: [13068] ['ip', 'route', 'del', '169.254.0.0/16']
    Jun  9 09:53:54 Xen3 SM: [13068]   pread SUCCESS
    Jun  9 09:53:54 Xen3 SM: [13068] ['ifconfig', 'xapi12', '169.254.0.1', 'netmask', '255.255.0.0']
    Jun  9 09:53:54 Xen3 SM: [13068]   pread SUCCESS
    Jun  9 09:53:54 Xen3 SM: [13068] ['ip', 'route', 'add', '169.254.0.0/16', 'dev', 'xapi12',
'src', '169.254.0.1']
    Jun  9 09:53:54 Xen3 SM: [13068]   pread SUCCESS
    Jun  9 09:53:54 Xen3 SM: [13071] ['ip', 'route', 'del', '169.254.0.0/16']
    Jun  9 09:53:54 Xen3 SM: [13071]   pread SUCCESS
    Jun  9 09:53:54 Xen3 SM: [13071] ['ifconfig', 'xapi12', '169.254.0.1', 'netmask', '255.255.0.0']
    Jun  9 09:53:54 Xen3 SM: [13071]   pread SUCCESS
    Jun  9 09:53:54 Xen3 SM: [13071] ['ip', 'route', 'add', '169.254.0.0/16', 'dev', 'xapi12',
'src', '169.254.0.1']
    Jun  9 09:53:54 Xen3 SM: [13071]   pread SUCCESS
    
    
    Jun  9 09:54:00 Xen3 SM: [13115] on-slave.multi: {'vgName':
    'VG_XenStorage-469b6dcd-8466-3d03-de0e-cc3983e1b6e2', 'lvName1':
    'VHD-633338a7-6c40-4aa6-b88e-c798b6fdc04d', 'action1':
    'deactivateNoRefcount', 'action2': 'cleanupLock', 'uuid2':
    '633338a7-6c40-4aa6-b88e-c798b6fdc04d', 'ns2':
    'lvm-469b6dcd-8466-3d03-de0e-cc3983e1b6e2'}
    Jun  9 09:54:00 Xen3 SM: [13115] LVMCache created for
    VG_XenStorage-469b6dcd-8466-3d03-de0e-cc3983e1b6e2
    Jun  9 09:54:00 Xen3 SM: [13115] on-slave.action 1: deactivateNoRefcount Jun
    9 09:54:00 Xen3 SM: [13115] LVMCache: will initialize now Jun  9 09:54:00
    Xen3 SM: [13115] LVMCache: refreshing Jun  9 09:54:00 Xen3 SM: [13115] ['/usr/sbin/lvs',
'--noheadings', '--units', 'b', '-o', '+lv_tags', '/dev/VG_XenStorage-469b6dcd-8466-3d03-de0e-cc3983e1b6e2']
    Jun  9 09:54:00 Xen3 SM: [13115]   pread SUCCESS
    Jun  9 09:54:00 Xen3 SM: [13115] ['/usr/sbin/lvchange', '-an',
    '/dev/VG_XenStorage-469b6dcd-8466-3d03-de0e-cc3983e1b6e2/VHD-633338a7-6c40-4
    aa6-b88e-c798b6fdc04d']
    Jun  9 09:54:00 Xen3 SM: [13115]   pread SUCCESS
    Jun  9 09:54:00 Xen3 SM: [13115] ['/sbin/dmsetup', 'status',
    'VG_XenStorage--469b6dcd--8466--3d03--de0e--cc3983e1b6e2-VHD--633338a7--6c40
    --4aa6--b88e--c798b6fdc04d']
    Jun  9 09:54:00 Xen3 SM: [13115]   pread SUCCESS
    Jun  9 09:54:00 Xen3 SM: [13115] on-slave.action 2: cleanupLock
    
    Jun  9 09:54:16 Xen3 SM: [13230] ['ip', 'route', 'del', '169.254.0.0/16']
    Jun  9 09:54:16 Xen3 SM: [13230]   pread SUCCESS
    Jun  9 09:54:16 Xen3 SM: [13230] ['ifconfig', 'xapi12', '169.254.0.1', 'netmask', '255.255.0.0']
    Jun  9 09:54:16 Xen3 SM: [13230]   pread SUCCESS
    Jun  9 09:54:16 Xen3 SM: [13230] ['ip', 'route', 'add', '169.254.0.0/16', 'dev', 'xapi12',
'src', '169.254.0.1']
    Jun  9 09:54:16 Xen3 SM: [13230]   pread SUCCESS
    Jun  9 09:54:19 Xen3 updatempppathd: [15446] The garbage collection routine
    returned: 0 Jun  9 09:54:23 Xen3 SM: [13277] ['ip', 'route', 'del', '169.254.0.0/16']
    Jun  9 09:54:23 Xen3 SM: [13277]   pread SUCCESS
    Jun  9 09:54:23 Xen3 SM: [13277] ['ifconfig', 'xapi12', '169.254.0.1', 'netmask', '255.255.0.0']
    Jun  9 09:54:23 Xen3 SM: [13277]   pread SUCCESS
    Jun  9 09:54:23 Xen3 SM: [13277] ['ip', 'route', 'add', '169.254.0.0/16', 'dev', 'xapi12',
'src', '169.254.0.1']
    Jun  9 09:54:23 Xen3 SM: [13277]   pread SUCCESS
    
    Jeremy
    
    
    -----Original Message-----
    From: Jeremy Peterson [mailto:jpeterson@acentek.net]
    Sent: Friday, June 9, 2017 9:53 AM
    To: users@cloudstack.apache.org
    Subject: RE: Recreating SystemVM's
    
    I am checking SMlog now on all hosts.
    
    Jeremy
    
    
    -----Original Message-----
    From: Rajani Karuturi [mailto:rajani@apache.org]
    Sent: Friday, June 9, 2017 9:00 AM
    To: Users <users@cloudstack.apache.org>
    Subject: Re: Recreating SystemVM's
    
    on xenserver log, did you check what is causing "
    HOST_CANNOT_ATTACH_NETWORK"?
    
    ~Rajani
    http://cloudplatform.accelerite.com/
    
    On Fri, Jun 9, 2017 at 7:00 PM, Jeremy Peterson <jpeterson@acentek.net>
    wrote:
    
    > 08:28:43        select * from vm_instance where name like 's-%' limit
    > 10000     7481 row(s) returned    0.000 sec / 0.032 sec
    >
    > All vm's 'state' returned Destoryed outside of the current vm 7873 
    > which is in a Stopped state but that goes Destroyed and a new get created.
    >
    > Any other suggestions?
    >
    > Jeremy
    >
    >
    > -----Original Message-----
    > From: Jeremy Peterson [mailto:jpeterson@acentek.net]
    > Sent: Thursday, June 8, 2017 12:47 AM
    > To: users@cloudstack.apache.org
    > Subject: Re: Recreating SystemVM's
    >
    > I'll make that change in the am.
    >
    > Today I put a host in maintence and rebooted because proxy and 
    > secstore vm were constantly being created on that host and still no
    change.
    >
    > Let you know tomorrow.
    >
    > Jeremy
    >
    >
    > Sent from my Verizon, Samsung Galaxy smartphone
    >
    >
    > -------- Original message --------
    > From: Rajani Karuturi <rajani@apache.org>
    > Date: 6/8/17 12:07 AM (GMT-06:00)
    > To: Users <users@cloudstack.apache.org>
    > Subject: Re: Recreating SystemVM's
    >
    > Did you check SMLog on xenserver?
    > unable to destroy task(com.xensource.xenapi.Task@256829a8) on
    > host(b34f086e-fabf-471e-9feb-8f54362d7d0f) due to You gave an invalid 
    > object reference.  The object may have recently been deleted.  The 
    > class parameter gives the type of reference given, and the handle 
    > parameter echoes the bad value given.
    >
    > Looks like Destroy of SSVM failed. What state is SSVM in? mark it as 
    > Destroyed in cloud DB and wait for cloudstack to create a new SSVM.
    >
    > ~Rajani
    > http://cloudplatform.accelerite.com/
    >
    > On Thu, Jun 8, 2017 at 1:11 AM, Jeremy Peterson 
    > <jpeterson@acentek.net>
    > wrote:
    >
    > > Probably agreed.
    > >
    > > But I ran toolstack restart on all hypervisors and v-3193 just tried 
    > > to create and fail along with s-5398.
    > >
    > > The PIF error went away. But VM's are still recreating
    > >
    > > https://pastebin.com/4n4xBgMT
    > >
    > > New log from this afternoon.
    > >
    > > My catalina.out is over 4GB
    > >
    > > Jeremy
    > >
    > >
    > > -----Original Message-----
    > > From: Makrand [mailto:makrandsanap@gmail.com]
    > > Sent: Wednesday, June 7, 2017 12:52 AM
    > > To: users@cloudstack.apache.org
    > > Subject: Re: Recreating SystemVM's
    > >
    > > Hi there,
    > >
    > > Looks more like hypervisor issue.
    > >
    > > Just run *xe-toolstack-restart* on hosts where these VMs are trying 
    > > to start or if you don't have too many hosts, better run on all 
    > > members including master. most of i/o related issues squared off by 
    > > toolstack bounce.
    > >
    > > --
    > > Makrand
    > >
    > >
    > > On Wed, Jun 7, 2017 at 3:01 AM, Jeremy Peterson 
    > > <jpeterson@acentek.net>
    > > wrote:
    > >
    > > > Ok so I pulled this from Sunday morning.
    > > >
    > > > https://pastebin.com/nCETw1sC
    > > >
    > > >
    > > > errorInfo: [HOST_CANNOT_ATTACH_NETWORK, 
    > > > OpaqueRef:65d0c844-bd70-81e9-4518-8809e1dc0ee7,
    > > > OpaqueRef:0093ac3f-9f3a-37e1-9cdb-581398d27ba2]
    > > >
    > > > XenServer error.
    > > >
    > > > Now this still gets me because all of the other VM's launched just
    > fine.
    > > >
    > > > Going into XenCenter I see an error at the bottom This PIF is a 
    > > > bond slave and cannot be plugged.
    > > >
    > > > ???
    > > >
    > > > If I go to networking on the hosts I see the storage vlans and 
    > > > bonds are all there.
    > > >
    > > > I see my GUEST-PUB bond is there and LACP is setup correct.
    > > >
    > > > Any suggestions ?
    > > >
    > > >
    > > > Jeremy
    > > >
    > > >
    > > > -----Original Message-----
    > > > From: Jeremy Peterson [mailto:jpeterson@acentek.net]
    > > > Sent: Tuesday, June 6, 2017 9:23 AM
    > > > To: users@cloudstack.apache.org
    > > > Subject: RE: Recreating SystemVM's
    > > >
    > > > Thank you all for those responses.
    > > >
    > > > I'll comb through my management-server.log and post a pastebin if 
    > > > I'm scratching my head.
    > > >
    > > > Jeremy
    > > >
    > > > -----Original Message-----
    > > > From: Rajani Karuturi [mailto:rajani@apache.org]
    > > > Sent: Tuesday, June 6, 2017 6:53 AM
    > > > To: users@cloudstack.apache.org
    > > > Subject: Re: Recreating SystemVM's
    > > >
    > > > If the zone is enabled, cloudstack should recreate them automatically.
    > > >
    > > > ~ Rajani
    > > >
    > > > http://cloudplatform.accelerite.com/
    > > >
    > > > On June 6, 2017 at 11:37 AM, Erik Weber (terbolous@gmail.com)
    > > > wrote:
    > > >
    > > > CloudStack should recreate automatically, check the mgmt server 
    > > > logs for hints of why it doesn't happen.
    > > >
    > > > --
    > > > Erik
    > > >
    > > > tir. 6. jun. 2017 kl. 04.29 skrev Jeremy Peterson
    > > > <jpeterson@acentek.net>:
    > > >
    > > > I had an issue Sunday morning with cloudstack 4.9.0 and xenserver
    > 6.5.0.
    > > > My hosts stop sending LACP PDU's and caused a network drop to 
    > > > iSCSI primary storage.
    > > >
    > > > So all my instances recovered via HA enabled.
    > > >
    > > > But my console proxy and secondary storage system VM's got stuck 
    > > > in a boot state that would not power on.
    > > >
    > > > At this time they are expunged and gone.
    > > >
    > > > How do I tell cloudstack-management to recreate system VM's?
    > > >
    > > > I'm drawing a blank since deploying CS two years ago and just 
    > > > keeping things running and adding hosts and more storage 
    > > > everything has been so stable.
    > > >
    > > > Jeremy
    > > >
    > >
    >
    
    
    
    - proIO GmbH -
    Geschäftsführer: Swen Brüseke
    Sitz der Gesellschaft: Frankfurt am Main
    
    USt-IdNr. DE 267 075 918
    Registergericht: Frankfurt am Main - HRB 86239
    
    Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. 
    Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben,

    informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. 
    Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail sind nicht gestattet.

    
    This e-mail may contain confidential and/or privileged information. 
    If you are not the intended recipient (or have received this e-mail in error) please notify

    the sender immediately and destroy this e-mail.  
    Any unauthorized copying, disclosure or distribution of the material in this e-mail is
strictly forbidden. 
    
    
    


daan.hoogland@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
  
 

Mime
View raw message