cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chandan Purushothama (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CLOUDSTACK-2127) Used Master Branch System VM Template for VMWare: Failed to Create Load Balancing Rule
Date Mon, 22 Apr 2013 00:35:15 GMT
Chandan Purushothama created CLOUDSTACK-2127:
------------------------------------------------

             Summary: Used Master Branch System VM Template for VMWare: Failed to Create Load
Balancing Rule
                 Key: CLOUDSTACK-2127
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2127
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: Management Server
    Affects Versions: 4.2.0
            Reporter: Chandan Purushothama
            Priority: Critical
             Fix For: 4.2.0


===============
Steps to Reproduce:
===============

1. Create a Load Balancing Rule where the Virtual Router is deployed using the new Master
System VM Template for VMWare

============
Observations:
============

------------------------------------------------
Create Load Balancing Rule Job:
------------------------------------------------

[root@asfmgmt ~]# grep -i "job-24" /var/log/cloudstack/management/management-server.log
2013-04-21 17:22:03,866 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-3:null) submit
async job-24, details: AsyncJobVO {id:24, userId: 3, accountId: 3, sessionKey: null, instanceType:
null, instanceId: null, cmd: org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd,
cmdOriginator: null, cmdInfo: {"response":"json","id":"0d569a49-ee02-4f32-93b5-31f1bc9aaa99","sessionkey":"LmU2BWQx2jc+w2FawY1qyY9foFk\u003d","virtualmachineids":"d68f7c60-d963-4a32-bcd4-1b5f37aaa992,3a5f350d-0740-424a-9172-7fb893effdeb","ctxUserId":"3","_":"1366590152786","ctxAccountId":"3","ctxStartEventId":"115"},
cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, processStatus: 0, resultCode:
0, result: null, initMsid: 7200344900649, completeMsid: null, lastUpdated: null, lastPolled:
null, created: null}
2013-04-21 17:22:03,870 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-30:job-24) Executing
org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd for job-24
2013-04-21 17:22:03,882 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-30:job-24) Sync
job-24 execution on object network.204
2013-04-21 17:22:03,898 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-30:job-24) job
org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd for job-24
was queued, processing the queue.
2013-04-21 17:22:03,904 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-30:job-24) Executing
sync queue item: SyncQueueItemVO {id:7, queueId: 1, contentType: AsyncJob, contentId: 24,
lastProcessMsid: 7200344900649, lastprocessNumber: 7, lastProcessTime: Sun Apr 21 17:22:03
PDT 2013, created: Sun Apr 21 17:22:03 PDT 2013}
2013-04-21 17:22:03,905 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-30:job-24) Schedule
queued job-24
2013-04-21 17:22:03,909 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-30:job-24)
There is a pending process in sync queue(id: 1)
2013-04-21 17:22:03,911 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-31:job-24) Executing
org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd for job-24
2013-04-21 17:22:03,944 DEBUG [cloud.user.AccountManagerImpl] (Job-Executor-31:job-24) Access
to Rule[4-LoadBalancing-Add] granted to Acct[3-atoms] by DomainChecker_EnhancerByCloudStack_64236710
2013-04-21 17:22:03,945 DEBUG [cloud.user.AccountManagerImpl] (Job-Executor-31:job-24) Access
to VM[User|Atoms-VM-2] granted to Acct[3-atoms] by DomainChecker_EnhancerByCloudStack_64236710
2013-04-21 17:22:03,947 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-31:job-24)
Adding VM[User|Atoms-VM-2] to the load balancer pool
2013-04-21 17:22:03,954 DEBUG [cloud.user.AccountManagerImpl] (Job-Executor-31:job-24) Access
to Rule[4-LoadBalancing-Add] granted to Acct[3-atoms] by DomainChecker_EnhancerByCloudStack_64236710
2013-04-21 17:22:03,955 DEBUG [cloud.user.AccountManagerImpl] (Job-Executor-31:job-24) Access
to VM[User|Atoms-VM-1] granted to Acct[3-atoms] by DomainChecker_EnhancerByCloudStack_64236710
2013-04-21 17:22:03,957 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-31:job-24)
Adding VM[User|Atoms-VM-1] to the load balancer pool
2013-04-21 17:22:04,109 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-31:job-24)
Applying ip association in network Ntwk[204|Guest|8]
2013-04-21 17:22:04,129 DEBUG [agent.transport.Request] (Job-Executor-31:job-24) Seq 1-1166875769:
Sending  { Cmd , MgmtId: 7200344900649, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":3,"publicIp":"10.223.139.70","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"1391","vlanGateway":"10.223.139.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:c6:4a:00:00:0c","networkRate":200,"trafficType":"Public"},{"accountId":3,"publicIp":"10.223.139.74","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"1391","vlanGateway":"10.223.139.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:ef:5a:00:00:10","networkRate":200,"trafficType":"Public"},{"accountId":3,"publicIp":"10.223.139.75","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"1391","vlanGateway":"10.223.139.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:89:90:00:00:11","networkRate":200,"trafficType":"Public"},{"accountId":3,"publicIp":"10.223.139.73","sourceNat":false,"add":true,"oneToOneNat":true,"firstIP":false,"vlanId":"1391","vlanGateway":"10.223.139.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:81:ce:00:00:0f","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"10.223.59.80","router.name":"r-4-SYSVMTMPL"},"wait":0}}]
}
2013-04-21 17:22:04,130 DEBUG [agent.transport.Request] (Job-Executor-31:job-24) Seq 1-1166875769:
Executing:  { Cmd , MgmtId: 7200344900649, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":3,"publicIp":"10.223.139.70","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"1391","vlanGateway":"10.223.139.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:c6:4a:00:00:0c","networkRate":200,"trafficType":"Public"},{"accountId":3,"publicIp":"10.223.139.74","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"1391","vlanGateway":"10.223.139.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:ef:5a:00:00:10","networkRate":200,"trafficType":"Public"},{"accountId":3,"publicIp":"10.223.139.75","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"1391","vlanGateway":"10.223.139.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:89:90:00:00:11","networkRate":200,"trafficType":"Public"},{"accountId":3,"publicIp":"10.223.139.73","sourceNat":false,"add":true,"oneToOneNat":true,"firstIP":false,"vlanId":"1391","vlanGateway":"10.223.139.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:81:ce:00:00:0f","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"10.223.59.80","router.name":"r-4-SYSVMTMPL"},"wait":0}}]
}
2013-04-21 17:22:22,917 DEBUG [agent.transport.Request] (Job-Executor-31:job-24) Seq 1-1166875769:
Received:  { Ans: , MgmtId: 7200344900649, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
2013-04-21 17:22:22,925 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-31:job-24)
Applying firewall rules in network Ntwk[204|Guest|8]
2013-04-21 17:22:22,965 DEBUG [cloud.network.NetworkModelImpl] (Job-Executor-31:job-24) Service
SecurityGroup is not supported in the network id=204
2013-04-21 17:22:23,002 DEBUG [agent.transport.Request] (Job-Executor-31:job-24) Seq 1-1166875771:
Sending  { Cmd , MgmtId: 7200344900649, via: 1, Ver: v1, Flags: 100001, [{"routing.LoadBalancerConfigCommand":{"loadBalancers":[{"uuid":"0d569a49-ee02-4f32-93b5-31f1bc9aaa99","srcIp":"10.223.139.75","srcPort":22,"protocol":"tcp","algorithm":"roundrobin","revoked":false,"alreadyAdded":false,"inline":false,"destinations":[{"destIp":"10.1.1.63","destPort":22,"revoked":false,"alreadyAdded":false},{"destIp":"10.1.1.146","destPort":22,"revoked":false,"alreadyAdded":false}]}],"lbStatsVisibility":"global","lbStatsPublicIP":"10.223.139.70","lbStatsPrivateIP":"10.223.59.80","lbStatsGuestIP":"10.1.1.1","lbStatsPort":"8081","lbStatsSrcCidrs":"0/0","lbStatsAuth":"admin1:AdMiN123","lbStatsUri":"/admin?stats","nic":{"deviceId":0,"networkRateMbps":200,"defaultNic":false,"uuid":"6e104794-6262-4516-b886-6130eabc1e5f","ip":"10.1.1.1","netmask":"255.255.255.0","mac":"02:00:40:dd:00:02","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2586","isolationUri":"vlan://2586","isSecurityGroupEnabled":false},"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"10.223.59.80","router.name":"r-4-SYSVMTMPL"},"wait":0}}]
}
2013-04-21 17:22:23,002 DEBUG [agent.transport.Request] (Job-Executor-31:job-24) Seq 1-1166875771:
Executing:  { Cmd , MgmtId: 7200344900649, via: 1, Ver: v1, Flags: 100001, [{"routing.LoadBalancerConfigCommand":{"loadBalancers":[{"uuid":"0d569a49-ee02-4f32-93b5-31f1bc9aaa99","srcIp":"10.223.139.75","srcPort":22,"protocol":"tcp","algorithm":"roundrobin","revoked":false,"alreadyAdded":false,"inline":false,"destinations":[{"destIp":"10.1.1.63","destPort":22,"revoked":false,"alreadyAdded":false},{"destIp":"10.1.1.146","destPort":22,"revoked":false,"alreadyAdded":false}]}],"lbStatsVisibility":"global","lbStatsPublicIP":"10.223.139.70","lbStatsPrivateIP":"10.223.59.80","lbStatsGuestIP":"10.1.1.1","lbStatsPort":"8081","lbStatsSrcCidrs":"0/0","lbStatsAuth":"admin1:AdMiN123","lbStatsUri":"/admin?stats","nic":{"deviceId":0,"networkRateMbps":200,"defaultNic":false,"uuid":"6e104794-6262-4516-b886-6130eabc1e5f","ip":"10.1.1.1","netmask":"255.255.255.0","mac":"02:00:40:dd:00:02","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2586","isolationUri":"vlan://2586","isSecurityGroupEnabled":false},"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"10.223.59.80","router.name":"r-4-SYSVMTMPL"},"wait":0}}]
}
2013-04-21 17:22:30,957 DEBUG [agent.transport.Request] (Job-Executor-31:job-24) Seq 1-1166875771:
Received:  { Ans: , MgmtId: 7200344900649, via: 1, Ver: v1, Flags: 0, { Answer } }
2013-04-21 17:22:30,961 WARN  [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-31:job-24)
Unable to apply the load balancer config because resource is unavaliable.
2013-04-21 17:22:30,963 INFO  [cloud.utils.AnnotationHelper] (Job-Executor-31:job-24)
2013-04-21 17:22:30,984 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-31:job-24) Unexpected
exception while executing org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd
2013-04-21 17:22:30,985 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-31:job-24) Complete
async job-24, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to
add specified loadbalancerruleid for vms [6, 3]
2013-04-21 17:22:30,996 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-31:job-24)
Sync queue (1) is currently empty
2013-04-21 17:22:31,092 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-25:null) Async
job-24 completed

================
ERROR:
================

2013-04-21 17:22:29,784 ERROR [utils.ssh.SshHelper] (DirectAgent-126:10.223.59.66) SSH execution
of command /root/loadbalancer.sh -i 10.223.59.80 -f /tmp/10_223_59_80.cfg -a 10.223.139.75:22:,
-s 10.223.139.70:8081:0/0:,, has an error status code in return. result output: mv: cannot
stat `/var/run/haproxy.pid': No such file or directory
cat: /var/run/haproxy.pid.old: No such file or directory
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
/root/reconfigLB.sh: line 28: haproxy: command not found
cat: /var/run/haproxy.pid.old: No such file or directory
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
mv: cannot stat `/var/run/haproxy.pid.old': No such file or directory

2013-04-21 17:22:29,791 ERROR [vmware.resource.VmwareResource] (DirectAgent-126:10.223.59.66)
LoadBalancerConfigCommand on domain router 10.223.59.80 failed. message: mv: cannot stat `/var/run/haproxy.pid':
No such file or directory
cat: /var/run/haproxy.pid.old: No such file or directory
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
/root/reconfigLB.sh: line 28: haproxy: command not found
cat: /var/run/haproxy.pid.old: No such file or directory
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
mv: cannot stat `/var/run/haproxy.pid.old': No such file or directory

2013-04-21 17:22:30,957 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-126:null) Seq
1-1166875771: Response Received:
2013-04-21 17:22:30,957 DEBUG [agent.transport.Request] (DirectAgent-126:null) Seq 1-1166875771:
Processing:  { Ans: , MgmtId: 7200344900649, via: 1, Ver: v1, Flags: 0, [{"Answer":{"result":false,"details":"LoadBalancerConfigCommand
on domain router 10.223.59.80 failed. message: mv: cannot stat `/var/run/haproxy.pid': No
such file or directory\ncat: /var/run/haproxy.pid.old: No such file or directory\nkill: usage:
kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]\n/root/reconfigLB.sh:
line 28: haproxy: command not found\ncat: /var/run/haproxy.pid.old: No such file or directory\nkill:
usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]\nmv:
cannot stat `/var/run/haproxy.pid.old': No such file or directory\n","wait":0}}] }
2013-04-21 17:22:30,957 DEBUG [agent.transport.Request] (Job-Executor-31:job-24) Seq 1-1166875771:
Received:  { Ans: , MgmtId: 7200344900649, via: 1, Ver: v1, Flags: 0, { Answer } }
2013-04-21 17:22:30,961 WARN  [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-31:job-24)
Unable to apply the load balancer config because resource is unavaliable.
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply firewall rules on router
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3439)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyFirewallRules(VirtualNetworkApplianceManagerImpl.java:3294)
        at com.cloud.network.element.VirtualRouterElement.applyLBRules(VirtualRouterElement.java:361)
        at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyRules(LoadBalancingRulesManagerImpl.java:1425)
        at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:2428)
        at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:1461)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:1385)
        at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:909)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:100)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:164)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
2013-04-21 17:22:30,963 INFO  [cloud.utils.AnnotationHelper] (Job-Executor-31:job-24)
class java.lang.Stringdoes not have a Table annotation

2013-04-21 17:22:30,984 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-31:job-24) Unexpected
exception while executing org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd
com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid
for vms [6, 3]
        at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:932)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.loadbalancer.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:100)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:164)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
2013-04-21 17:22:30,985 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-31:job-24) Complete
async job-24, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to
add specified loadbalancerruleid for vms [6, 3]
2013-04-21 17:22:30,996 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-31:job-24)
Sync queue (1) is currently empty


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message