cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jayapal Reddy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-7396) [Automation] Failed to stop VPC router in KVM
Date Fri, 22 Aug 2014 07:01:15 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106563#comment-14106563
] 

Jayapal Reddy commented on CLOUDSTACK-7396:
-------------------------------------------

In the kvm host libvirt is not running:
[root@Rack2Host12 ~]# service libvirtd status
libvirtd dead but pid file exists
[root@Rack2Host12 ~]# ps aux | grep libvirtd
root     14238  0.0  0.0 103212   820 pts/13   S+   22:56   0:00 grep libvirtd
[root@Rack2Host12 ~]#

In libvirtd logs I saw error logs related to primary storage.
I started libvirtd and it is running successfully now. 
Now the VR stop should success.

*agent logs:
2014-08-21 22:54:04,848 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) Processing
command: com.cloud.agent.api.routing.SetNetworkACLCommand
2014-08-21 22:54:04,848 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null)
Executing: /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh vpc_acl.sh 169.254.3.25
 -d eth3 -M 02:00:0e:b0:00:02 -i 10.1.2.1 -m 24 -a Ingress:reverted:0:0:0:,
2014-08-21 22:54:05,132 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-3:null)
Execution is successful.
2014-08-21 22:54:05,132 DEBUG [cloud.agent.Agent] (agentRequest-Handler-3:null) Seq 2-3001649151642437575:
 { Ans: , MgmtId: 29066118877352, via: 2, Ver: v1, Flags: 0, [{"com.cloud.agent.api.Answer":{"result":true,"details":"","wait":0}}]
}
2014-08-21 22:54:05,158 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) Request:Seq
2-3001649151642437576:  { Cmd , MgmtId: 29066118877352, via: 2, Ver: v1, Flags: 100111, [{"com.cloud.agent.api.SetupGuestNetworkCommand":{"dhcpRange":"10.1.2.1","networkDomain":"test.domain.org","isRedundant":false,"add":false,"nic":{"deviceId":3,"networkRateMbps":200,"defaultNic":false,"pxeDisable":true,"nicUuid":"bc8b9a9f-7c76-4a73-80d5-444972c6122a","uuid":"ab491a32-d28b-4010-9da9-ab5db94a8744","ip":"10.1.2.1","netmask":"255.255.255.0","gateway":"10.1.2.1","mac":"02:00:0e:b0:00:02","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2303","isolationUri":"vlan://2303","isSecurityGroupEnabled":false},"accessDetails":{"router.guest.ip":"10.1.2.1","guest.vlan.tag":"2303","guest.network.gateway":"10.1.2.1","guest.bridge":"10.1.2.255","router.ip":"169.254.3.25","router.name":"r-918-VM"},"wait":0}}]
}
2014-08-21 22:54:05,158 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) Processing
command: com.cloud.agent.api.SetupGuestNetworkCommand
2014-08-21 22:54:05,159 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null)
Connection with libvirtd is broken, due to Cannot write data: Broken pipe
2014-08-21 22:54:05,159 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null)
can't find connection: KVM, for vm: r-918-VM, continue
2014-08-21 22:54:05,159 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null)
Connection with libvirtd is broken, due to Cannot write data: Broken pipe
2014-08-21 22:54:05,159 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null)
can't find connection: LXC, for vm: r-918-VM, continue
2014-08-21 22:54:05,160 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null)
can't find which hypervisor the vm used , then use the default hypervisor
2014-08-21 22:54:05,160 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-4:null)
Connection with libvirtd is broken, due to Cannot write data: Broken pipe
2014-08-21 22:54:05,160 WARN  [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-4:null)
Creating guest network failed due to org.libvirt.LibvirtException: Failed to connect socket
to '/var/run/libvirt/libvirt-sock': Connection refused
org.libvirt.LibvirtException: Failed to connect socket to '/var/run/libvirt/libvirt-sock':
Connection refused
        at org.libvirt.ErrorHandler.processError(Unknown Source)
        at org.libvirt.Connect.processError(Unknown Source)
        at org.libvirt.Connect.processError(Unknown Source)
        at org.libvirt.Connect.<init>(Unknown Source)
        at com.cloud.hypervisor.kvm.resource.LibvirtConnection.getConnection(LibvirtConnection.java:51)
        at com.cloud.hypervisor.kvm.resource.LibvirtConnection.getConnection(LibvirtConnection.java:37)
        at com.cloud.hypervisor.kvm.resource.LibvirtConnection.getConnectionByVmName(LibvirtConnection.java:75)
        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.prepareNetworkElementCommand(LibvirtComputingResource.java:2115)
        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.prepareCommand(LibvirtComputingResource.java:381)
        at com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.executeRequest(VirtualRoutingResource.java:96)
        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1338)
        at com.cloud.agent.Agent.processRequest(Agent.java:503)
        at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:810)
        at com.cloud.utils.nio.Task.run(Task.java:84)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)
2014-08-21 22:54:05,160 ERROR [resource.virtualnetwork.VirtualRoutingResource] (agentRequest-Handler-4:null)
Failed to prepare VR command due to Creating guest network failed due to org.libvirt.LibvirtException:
Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
2014-08-21 22:54:05,160 DEBUG [cloud.agent.Agent] (agentRequest-Handler-4:null) Seq 2-3001649151642437576:
 { Ans: , MgmtId: 29066118877352, via: 2, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":false,"details":"Creating
guest network failed due to org.libvirt.LibvirtException: Failed to connect socket to '/var/run/libvirt/libvirt-sock':
Connection refused","wait":0}}] }
~

> [Automation] Failed to stop VPC router in KVM
> ---------------------------------------------
>
>                 Key: CLOUDSTACK-7396
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7396
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Automation
>    Affects Versions: 4.5.0
>            Reporter: Rayees Namathponnan
>            Assignee: Rayees Namathponnan
>            Priority: Blocker
>             Fix For: 4.5.0
>
>         Attachments: agent.rar, kvm.sql, management-server.rar
>
>
> This issue observed with latest automation run,  deploy a VPC router and stop 
> Virtual router stop fails with below exception 
> {noformat}
> lhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAC-XQAGVZpcn
> R1YWxNYWNoaW5lTWFuYWdlckltcGwA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0,
resultCode: 0, result: null, initMsid:
>  29066118877352, completeMsid: null, lastUpdated: null, lastPolled: null, created: Thu
Aug 21 22:24:10 PDT 2014}, job origin
> :8131
> com.cloud.utils.exception.CloudRuntimeException: Unable to stop VM[DomainRouter|r-761-VM]
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:1523)
>         at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStop(VirtualMachineManagerImpl.java:1377)
>         at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStop(VirtualMachineManagerImpl.java:4594)
>         at sun.reflect.GeneratedMethodAccessor162.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>         at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4738)
>         at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:744)
> 2014-08-21 22:24:11,959 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-28:ctx-8824da69
job-8131/job-8162) Comple
> te async job-8162, jobStatus: FAILED, resultCode: 0, result: rO0ABXNyABpqYXZhLmxhbmcuUnVudGltZUV4Y2VwdGlvbp5fBkcKNIPlAgAAeHI
> AE2phdmEubGFuZy5FeGNlcHRpb27Q_R8-GjscxAIAAHhyABNqYXZhLmxhbmcuVGhyb3dhYmxl1cY1Jzl3uMsDAARMAAVjYXVzZXQAFUxqYXZhL2xhbmcvVGhyb3d
> hYmxlO0wADWRldGFpbE1lc3NhZ2V0ABJMamF2YS9sYW5nL1N0cmluZztbAApzdGFja1RyYWNldAAeW0xqYXZhL2xhbmcvU3RhY2tUcmFjZUVsZW1lbnQ7TAAUc3V
> wcHJlc3NlZEV4Y2VwdGlvbnN0ABBMamF2YS91dGlsL0xpc3Q7eHBxAH4AB3QAREpvYiBmYWlsZWQgZHVlIHRvIGV4Y2VwdGlvbiBVbmFibGUgdG8gc3RvcCBWTVt
> Eb21haW5Sb3V0ZXJ8ci03NjEtVk1ddXIAHltMamF2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50OwJGKjw8_SI5AgAAeHAAAAANc3IAG2phdmEubGFuZy5TdGFja1R
> yYWNlRWxlbWVudGEJxZomNt2FAgAESQAKbGluZU51bWJlckwADmRlY2xhcmluZ0NsYXNzcQB-AARMAAhmaWxlTm
> 2014-08-21 22:42:19,885 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-149:ctx-5eeadd75
job-8131/job-8304) Unable to c
> omplete AsyncJobVO {id:8304, userId: 1, accountId: 1, instanceType: null, instanceId:
null, cmd: com.cloud.vm.VmWorkStop, cm
> dInfo: rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKA
> AlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAADBnQAGVZpc
> nR1YWxNYWNoaW5lTWFuYWdlckltcGwA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0,
resultCode: 0, result: null, initMsid
> : 29066118877352, completeMsid: null, lastUpdated: null, lastPolled: null, created: Thu
Aug 21 22:42:18 PDT 2014}, job origi
> n:8131
> com.cloud.utils.exception.CloudRuntimeException: Unable to stop VM[DomainRouter|r-774-VM]
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:1523)
>         at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStop(VirtualMachineManagerImpl.java:1377)
>         at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStop(VirtualMachineManagerImpl.java:4594)
>         at sun.reflect.GeneratedMethodAccessor162.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>         at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4738)
>         at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:744)
> 2014-08-21 22:42:19,886 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-149:ctx-5eeadd75
job-8131/job-8304) Complete async job-8304, jobStatus: FAILED, resultCode: 0, result: rO0ABXNyABpqYXZhLmxhbmcuUnVudGltZUV4Y2VwdGlvbp5fBkcKNIPlAgAAeHIAE2phdmEubGFuZy5FeGNlcHRpb27Q_R8-GjscxAIAAHhyABNqYXZhLmxhbmcuVGhyb3dhYmxl1cY1Jzl3uMsDAARMAAVjYXVzZXQAFUxqYXZhL2xhbmcvVGhyb3dhYmxlO0wADWRldGFpbE1lc3NhZ2V0ABJMamF2YS9sYW5nL1N0cmluZztbAApzdGFja1RyYWNldAAeW0xqY
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message