cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "manasaveloori (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CLOUDSTACK-8927) [VPC]Executing command in VR: /opt/cloud/bin/router_proxy.sh is failing whenever there is a configuration change in VR
Date Thu, 01 Oct 2015 04:37:04 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

manasaveloori updated CLOUDSTACK-8927:
--------------------------------------
    Description: 
Whenever there is a configuration change in VPC VR observing the connectivity issues with
VR.

Case1:
Created VPC and tier network with default allow.
Now created a new ACL list and rules. Changed the ACL list for the tier network.


2015-09-30 04:35:39,553 ERROR [c.c.u.s.SshHelper] (DirectAgent-336:ctx-b9e5cdf1) SSH execution
of command /opt/cloud/bin/router_proxy.sh update_config.py 169.254.3.89 guest_network.json
has an error status code in return. result output:
2015-09-30 04:35:39,554 DEBUG [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-336:ctx-b9e5cdf1)
Processing ScriptConfigItem, executing update_config.py guest_network.json took 21165ms
2015-09-30 04:35:39,554 WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-336:ctx-b9e5cdf1)
Expected 1 answers while executing SetupGuestNetworkCommand but received 2


2015-09-30 04:35:45,769 ERROR [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-94:ctx-56b18174
job-227/job-228 ctx-f92247d7) Failed to start instance VM[DomainRouter|r-22-VM]
com.cloud.utils.exception.ExecutionException: Unable to start VM[DomainRouter|r-22-VM] due
to error in finalizeStart, not retrying
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1083)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4576)
        at sun.reflect.GeneratedMethodAccessor382.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
        at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4732)
        at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:537)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.Def

Case2:

Reboot VR with remote access VPN enabled on VPC VR:

Created VPC ,enabled vpn and rebooted the VR.
ERROR in logs:

2015-09-30 04:46:18,663 ERROR [c.c.u.s.SshHelper] (DirectAgent-46:ctx-3c355a22) SSH execution
of command /opt/cloud/bin/router_proxy.sh update_config.py 169.254.0.95 vpn_user_list.json
has an error status code in return. result output:
2015-09-30 04:46:18,664 DEBUG [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-46:ctx-3c355a22)
Processing ScriptConfigItem, executing update_config.py vpn_user_list.json took 21168ms
2015-09-30 04:46:18,664 WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-46:ctx-3c355a22)
Expected 1 answers while executing VpnUsersCfgCommand but received 2




015-09-30 04:46:24,821 ERROR [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-101:ctx-fecf4919
job-240/job-242 ctx-44fde71b) Failed to start instance VM[DomainRouter|r-23-VM]
com.cloud.utils.exception.ExecutionException: Unable to start VM[DomainRouter|r-23-VM] due
to error in finalizeStart, not retrying
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1083)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4576)
        at sun.reflect.GeneratedMethodAccessor382.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
        at com.cloud.vm.VirtualMach


Case3:
Created site to site VPN
Enable vpn on VPCA and then enabling VPN on VPCB is throwing following error:
Observed the following error:

2015-09-29 12:17:40,600 ERROR [c.c.u.s.SshHelper] (DirectAgent-392:ctx-d373204a) Timed out
in waiting SSH execution result
2015-09-29 12:17:40,607 DEBUG [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-392:ctx-d373204a)
Processing ScriptConfigItem, executing update_config.py site_2_site_vpn.json took 120148ms
2015-09-29 12:17:40,607 WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-392:ctx-d373204a)
Expected 1 answers while executing Site2SiteVpnCfgCommand but received 2
2015-09-29 12:17:40,607 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-392:ctx-d373204a)
Seq 4-50384020831211212: Response Received:
2015-09-29 12:17:40,608 DEBUG [c.c.a.t.Request] (DirectAgent-392:ctx-d373204a) Seq 4-50384020831211212:
Processing:  { Ans: , MgmtId: 6877316382886, via: 4, Ver: v1, Flags: 100, [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null
- success: ","null - failed: Timed out in waiting SSH execution result"],"result":false,"wait":0}}]
}
2015-09-29 12:17:40,608 DEBUG [c.c.a.t.Request] (API-Job-Executor-94:ctx-cc4f9605 job-194
ctx-8bccbe8f) Seq 4-50384020831211212: Received:  { Ans: , MgmtId: 6877316382886, via: 4,
Ver: v1, Flags: 100, { GroupAnswer } }
2015-09-29 12:17:40,610 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-392:ctx-d373204a) Seq 4-50384020831211212:
No more commands found
2015-09-29 12:17:40,644 WARN  [o.a.c.a.c.u.v.CreateVpnConnectionCmd] (API-Job-Executor-94:ctx-cc4f9605
job-194 ctx-8bccbe8f) Exception:
com.cloud.exception.ResourceUnavailableException: Resource [Site2SiteVpnConnection:1] is unreachable:
Failed to apply site-to-site VPN
        at com.cloud.network.vpn.Site2SiteVpnManagerImpl.startVpnConnection(Site2SiteVpnManagerImpl.java:345)




  was:
Whenever there is a configuration change in VPC VR observing the connectivity issues with
VR.

Case1:
Created VPC and tier network with default allow.
Now created a new ACL list and rules. Changed the ACL list for the tier network.


2015-09-30 04:35:39,553 ERROR [c.c.u.s.SshHelper] (DirectAgent-336:ctx-b9e5cdf1) SSH execution
of command /opt/cloud/bin/router_proxy.sh update_config.py 169.254.3.89 guest_network.json
has an error status code in return. result output:
2015-09-30 04:35:39,554 DEBUG [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-336:ctx-b9e5cdf1)
Processing ScriptConfigItem, executing update_config.py guest_network.json took 21165ms
2015-09-30 04:35:39,554 WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-336:ctx-b9e5cdf1)
Expected 1 answers while executing SetupGuestNetworkCommand but received 2


2015-09-30 04:35:45,769 ERROR [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-94:ctx-56b18174
job-227/job-228 ctx-f92247d7) Failed to start instance VM[DomainRouter|r-22-VM]
com.cloud.utils.exception.ExecutionException: Unable to start VM[DomainRouter|r-22-VM] due
to error in finalizeStart, not retrying
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1083)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4576)
        at sun.reflect.GeneratedMethodAccessor382.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
        at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4732)
        at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:537)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.Def

Case2:

Reboot VR with remote access VPN enabled on VPC VR:

Created VPC ,enabled vpn and rebooted the VR.
ERROR in logs:

2015-09-30 04:46:18,663 ERROR [c.c.u.s.SshHelper] (DirectAgent-46:ctx-3c355a22) SSH execution
of command /opt/cloud/bin/router_proxy.sh update_config.py 169.254.0.95 vpn_user_list.json
has an error status code in return. result output:
2015-09-30 04:46:18,664 DEBUG [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-46:ctx-3c355a22)
Processing ScriptConfigItem, executing update_config.py vpn_user_list.json took 21168ms
2015-09-30 04:46:18,664 WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-46:ctx-3c355a22)
Expected 1 answers while executing VpnUsersCfgCommand but received 2




015-09-30 04:46:24,821 ERROR [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-101:ctx-fecf4919
job-240/job-242 ctx-44fde71b) Failed to start instance VM[DomainRouter|r-23-VM]
com.cloud.utils.exception.ExecutionException: Unable to start VM[DomainRouter|r-23-VM] due
to error in finalizeStart, not retrying
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1083)
        at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4576)
        at sun.reflect.GeneratedMethodAccessor382.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
        at com.cloud.vm.VirtualMach


Case3:
Created site to site VPN
When trying to enable VPN on VPC A
Observed the following error:

2015-09-29 12:17:40,600 ERROR [c.c.u.s.SshHelper] (DirectAgent-392:ctx-d373204a) Timed out
in waiting SSH execution result
2015-09-29 12:17:40,607 DEBUG [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-392:ctx-d373204a)
Processing ScriptConfigItem, executing update_config.py site_2_site_vpn.json took 120148ms
2015-09-29 12:17:40,607 WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-392:ctx-d373204a)
Expected 1 answers while executing Site2SiteVpnCfgCommand but received 2
2015-09-29 12:17:40,607 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-392:ctx-d373204a)
Seq 4-50384020831211212: Response Received:
2015-09-29 12:17:40,608 DEBUG [c.c.a.t.Request] (DirectAgent-392:ctx-d373204a) Seq 4-50384020831211212:
Processing:  { Ans: , MgmtId: 6877316382886, via: 4, Ver: v1, Flags: 100, [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null
- success: ","null - failed: Timed out in waiting SSH execution result"],"result":false,"wait":0}}]
}
2015-09-29 12:17:40,608 DEBUG [c.c.a.t.Request] (API-Job-Executor-94:ctx-cc4f9605 job-194
ctx-8bccbe8f) Seq 4-50384020831211212: Received:  { Ans: , MgmtId: 6877316382886, via: 4,
Ver: v1, Flags: 100, { GroupAnswer } }
2015-09-29 12:17:40,610 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-392:ctx-d373204a) Seq 4-50384020831211212:
No more commands found
2015-09-29 12:17:40,644 WARN  [o.a.c.a.c.u.v.CreateVpnConnectionCmd] (API-Job-Executor-94:ctx-cc4f9605
job-194 ctx-8bccbe8f) Exception:
com.cloud.exception.ResourceUnavailableException: Resource [Site2SiteVpnConnection:1] is unreachable:
Failed to apply site-to-site VPN
        at com.cloud.network.vpn.Site2SiteVpnManagerImpl.startVpnConnection(Site2SiteVpnManagerImpl.java:345)





> [VPC]Executing command in VR: /opt/cloud/bin/router_proxy.sh is failing whenever there
is a configuration change in VR
> ----------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-8927
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8927
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Network Controller
>    Affects Versions: 4.6.0
>            Reporter: manasaveloori
>            Priority: Blocker
>             Fix For: 4.6.0
>
>         Attachments: management-server.rar, management-server.site-site.gz
>
>
> Whenever there is a configuration change in VPC VR observing the connectivity issues
with VR.
> Case1:
> Created VPC and tier network with default allow.
> Now created a new ACL list and rules. Changed the ACL list for the tier network.
> 2015-09-30 04:35:39,553 ERROR [c.c.u.s.SshHelper] (DirectAgent-336:ctx-b9e5cdf1) SSH
execution of command /opt/cloud/bin/router_proxy.sh update_config.py 169.254.3.89 guest_network.json
has an error status code in return. result output:
> 2015-09-30 04:35:39,554 DEBUG [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-336:ctx-b9e5cdf1)
Processing ScriptConfigItem, executing update_config.py guest_network.json took 21165ms
> 2015-09-30 04:35:39,554 WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-336:ctx-b9e5cdf1)
Expected 1 answers while executing SetupGuestNetworkCommand but received 2
> 2015-09-30 04:35:45,769 ERROR [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-94:ctx-56b18174
job-227/job-228 ctx-f92247d7) Failed to start instance VM[DomainRouter|r-22-VM]
> com.cloud.utils.exception.ExecutionException: Unable to start VM[DomainRouter|r-22-VM]
due to error in finalizeStart, not retrying
>         at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1083)
>         at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4576)
>         at sun.reflect.GeneratedMethodAccessor382.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>         at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4732)
>         at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:537)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>         at org.apache.cloudstack.managed.context.impl.Def
> Case2:
> Reboot VR with remote access VPN enabled on VPC VR:
> Created VPC ,enabled vpn and rebooted the VR.
> ERROR in logs:
> 2015-09-30 04:46:18,663 ERROR [c.c.u.s.SshHelper] (DirectAgent-46:ctx-3c355a22) SSH execution
of command /opt/cloud/bin/router_proxy.sh update_config.py 169.254.0.95 vpn_user_list.json
has an error status code in return. result output:
> 2015-09-30 04:46:18,664 DEBUG [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-46:ctx-3c355a22)
Processing ScriptConfigItem, executing update_config.py vpn_user_list.json took 21168ms
> 2015-09-30 04:46:18,664 WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-46:ctx-3c355a22)
Expected 1 answers while executing VpnUsersCfgCommand but received 2
> 015-09-30 04:46:24,821 ERROR [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-101:ctx-fecf4919
job-240/job-242 ctx-44fde71b) Failed to start instance VM[DomainRouter|r-23-VM]
> com.cloud.utils.exception.ExecutionException: Unable to start VM[DomainRouter|r-23-VM]
due to error in finalizeStart, not retrying
>         at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1083)
>         at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4576)
>         at sun.reflect.GeneratedMethodAccessor382.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>         at com.cloud.vm.VirtualMach
> Case3:
> Created site to site VPN
> Enable vpn on VPCA and then enabling VPN on VPCB is throwing following error:
> Observed the following error:
> 2015-09-29 12:17:40,600 ERROR [c.c.u.s.SshHelper] (DirectAgent-392:ctx-d373204a) Timed
out in waiting SSH execution result
> 2015-09-29 12:17:40,607 DEBUG [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-392:ctx-d373204a)
Processing ScriptConfigItem, executing update_config.py site_2_site_vpn.json took 120148ms
> 2015-09-29 12:17:40,607 WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-392:ctx-d373204a)
Expected 1 answers while executing Site2SiteVpnCfgCommand but received 2
> 2015-09-29 12:17:40,607 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-392:ctx-d373204a)
Seq 4-50384020831211212: Response Received:
> 2015-09-29 12:17:40,608 DEBUG [c.c.a.t.Request] (DirectAgent-392:ctx-d373204a) Seq 4-50384020831211212:
Processing:  { Ans: , MgmtId: 6877316382886, via: 4, Ver: v1, Flags: 100, [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null
- success: ","null - failed: Timed out in waiting SSH execution result"],"result":false,"wait":0}}]
}
> 2015-09-29 12:17:40,608 DEBUG [c.c.a.t.Request] (API-Job-Executor-94:ctx-cc4f9605 job-194
ctx-8bccbe8f) Seq 4-50384020831211212: Received:  { Ans: , MgmtId: 6877316382886, via: 4,
Ver: v1, Flags: 100, { GroupAnswer } }
> 2015-09-29 12:17:40,610 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-392:ctx-d373204a) Seq
4-50384020831211212: No more commands found
> 2015-09-29 12:17:40,644 WARN  [o.a.c.a.c.u.v.CreateVpnConnectionCmd] (API-Job-Executor-94:ctx-cc4f9605
job-194 ctx-8bccbe8f) Exception:
> com.cloud.exception.ResourceUnavailableException: Resource [Site2SiteVpnConnection:1]
is unreachable: Failed to apply site-to-site VPN
>         at com.cloud.network.vpn.Site2SiteVpnManagerImpl.startVpnConnection(Site2SiteVpnManagerImpl.java:345)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message