cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cs user <acldstk...@gmail.com>
Subject Re: Virtual Routers not responding to dns requests
Date Fri, 27 Mar 2015 15:31:11 GMT
Just to add, in the database, the  nics table:

isolation_uri used to be set to "ec2://untagged"
 for vm_type="DomainRouter" of mode Dhcp.

This is no longer happening however, they are just set as NULL. I don't see
a way to set this in the networks table however. How does this
isolation_uri column get set, is it from the database or hard coded in the
code?





On Fri, Mar 27, 2015 at 10:02 AM, cs user <acldstkusr@gmail.com> wrote:

> Hi Again... :-)
>
> So it looks like the vif on the xenserver is not being setup correctly (or
> has been removed). The following rule is defined on xen host which the
> broken router is running on:
>
> Chain r-10864-VM (4 references)
>>
>> target     prot opt source               destination
>>
>> RETURN     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-in vif718.0 --physdev-is-bridged
>>
>> RETURN     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-in vif718.1 --physdev-is-bridged
>>
>> ACCEPT     all  --  anywhere             anywhere
>>
>>
>>>
> The vif that is mentioned above is not present on the host. As below. But
> on the working router, the vif in the iptable rule does exist.
>
>
> On the host, we also see the following in the logs with the vif mentioned:
>
> Mar 26 14:04:31 xen011 script-vif: vif718.0: writing
> backend/vif/718/0/hotplug-status=connected
> Mar 26 14:04:31 xen011 scripts-vif: Setting vif718.1 MTU 1500
> Mar 26 14:04:31 xen011 scripts-vif: Adding vif718.1 to xapi4 with address
> fe:ff:ff:ff:ff:ff
> Mar 26 14:04:31 xen011 scripts-vif: Failed to ip link set vif718.1 address
> fe:ff:ff:ff:ff:ff
> Mar 26 14:04:31 xen011 python:
> /opt/xensource/libexec/setup-vif-rules[3233] - ['/sbin/ip', 'link', 'set',
> 'vif718.1', 'down']
> Mar 26 14:04:31 xen011 python:
> /opt/xensource/libexec/setup-vif-rules[3233] - ['/sbin/ebtables', '-L',
> 'FORWARD_vif718.1']
> Mar 26 14:04:31 xen011 python:
> /opt/xensource/libexec/setup-vif-rules[3233] - ['/sbin/ip', 'link', 'set',
> 'vif718.1', 'up']
> Mar 26 14:04:31 xen011 script-vif: vif718.1: writing
> backend/vif/718/1/hotplug-status=connected
> Mar 26 14:05:12 xen011 script-vif: vif718.1: removing
> backend/vif/718/1/hotplug-status
> Mar 26 14:05:12 xen011 script-vif: vif718.1: removing
> /xapi/718/hotplug/vif/1/hotplug
> Mar 26 14:05:12 xen011 scripts-vif: vif718.1 has been removed
> Mar 26 14:05:12 xen011 python:
> /opt/xensource/libexec/setup-vif-rules[4113] - ['/sbin/ip', 'link', 'set',
> 'vif718.1', 'down']
> Mar 26 14:05:12 xen011 python:
> /opt/xensource/libexec/setup-vif-rules[4113] - ['/sbin/ebtables', '-L',
> 'FORWARD_vif718.1']
> Mar 26 14:05:13 xen011 python:
> /opt/xensource/libexec/setup-vif-rules[4113] - ['/sbin/ip', 'link', 'set',
> 'vif718.1', 'up']
> Mar 26 14:05:13 xen011 script-vif: vif718.0: removing
> backend/vif/718/0/hotplug-status
> Mar 26 14:05:13 xen011 script-vif: vif718.0: removing
> /xapi/718/hotplug/vif/0/hotplug
> Mar 26 14:05:13 xen011 scripts-vif: vif718.0 has been removed
> Mar 26 14:05:13 xen011 python:
> /opt/xensource/libexec/setup-vif-rules[4156] - ['/sbin/ip', 'link', 'set',
> 'vif718.0', 'down']
> Mar 26 14:05:13 xen011 python:
> /opt/xensource/libexec/setup-vif-rules[4156] - ['/sbin/ebtables', '-L',
> 'FORWARD_vif718.0']
> Mar 26 14:05:13 xen011 python:
> /opt/xensource/libexec/setup-vif-rules[4156] - ['/sbin/ip', 'link', 'set',
> 'vif718.0', 'up']
> Mar 26 14:05:24 xen011 fe: 4917 (/sbin/ip addr show dev vif718.0) exitted
> with code 255
> Mar 26 14:05:25 xen011 fe: 5062 (/sbin/ip addr show dev vif718.1) exitted
> with code 255
>
>
> List of vif's, 718 is missing now however:
>
>
>> vif477.0  Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>>
>>          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
>>
>>          RX packets:150128316 errors:0 dropped:0 overruns:0 frame:0
>>
>>          TX packets:163423985 errors:0 dropped:0 overruns:0 carrier:0
>>
>>          collisions:0 txqueuelen:32
>>
>>          RX bytes:598157233 (570.4 MiB)  TX bytes:501933888 (478.6 MiB)
>>
>>
>>> vif671.0  Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>>
>>          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
>>
>>          RX packets:38112 errors:0 dropped:0 overruns:0 frame:0
>>
>>          TX packets:71566 errors:0 dropped:0 overruns:0 carrier:0
>>
>>          collisions:0 txqueuelen:32
>>
>>          RX bytes:2005682 (1.9 MiB)  TX bytes:92870677 (88.5 MiB)
>>
>>
>>> vif696.0  Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>>
>>          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
>>
>>          RX packets:20049 errors:0 dropped:0 overruns:0 frame:0
>>
>>          TX packets:49817 errors:0 dropped:0 overruns:0 carrier:0
>>
>>          collisions:0 txqueuelen:32
>>
>>          RX bytes:1215219 (1.1 MiB)  TX bytes:62130987 (59.2 MiB)
>>
>>
>>> vif703.0  Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>>
>>          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
>>
>>          RX packets:1459 errors:0 dropped:0 overruns:0 frame:0
>>
>>          TX packets:1803 errors:0 dropped:0 overruns:0 carrier:0
>>
>>          collisions:0 txqueuelen:32
>>
>>          RX bytes:48244 (47.1 KiB)  TX bytes:213662 (208.6 KiB)
>>
>>
>>> vif719.0  Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>>
>>          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
>>
>>          RX packets:1571 errors:0 dropped:0 overruns:0 frame:0
>>
>>          TX packets:75983 errors:0 dropped:2 overruns:0 carrier:0
>>
>>          collisions:0 txqueuelen:32
>>
>>          RX bytes:74416 (72.6 KiB)  TX bytes:3710662 (3.5 MiB)
>>
>>
>>> vif719.1  Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>>
>>          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
>>
>>          RX packets:7982 errors:0 dropped:0 overruns:0 frame:0
>>
>>          TX packets:8513 errors:0 dropped:1 overruns:0 carrier:0
>>
>>          collisions:0 txqueuelen:32
>>
>>          RX bytes:1349032 (1.2 MiB)  TX bytes:787782 (769.3 KiB)
>>
>>
>>> vif720.0  Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>>
>>          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
>>
>>          RX packets:75 errors:0 dropped:0 overruns:0 frame:0
>>
>>          TX packets:77 errors:0 dropped:0 overruns:0 carrier:0
>>
>>          collisions:0 txqueuelen:32
>>
>>          RX bytes:3404 (3.3 KiB)  TX bytes:5502 (5.3 KiB)
>>
>>
>
>
>
> On Fri, Mar 27, 2015 at 8:57 AM, cs user <acldstkusr@gmail.com> wrote:
>
>> Hi Jayapal,
>>
>> Those two parameters are both set to 1.
>>
>> We have a router which has survived the upgrade, and is still able to
>> receive and return dns requests from instances. We have checked on the
>> xenserver host and can see the following iptable config:
>>
>> Chain FORWARD (policy ACCEPT)
>>> target     prot opt source               destination
>>> BRIDGE-FIREWALL  all  --  anywhere             anywhere
>>>  PHYSDEV match --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth0+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out bond3+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out bond0+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth1+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth3+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth6+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out bond1+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth2+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth5+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth7+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth4+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out bond2+ --physdev-is-bridged
>>> DROP       all  --  anywhere             anywhere
>>
>>
>>
>> However, on a xenserver which is running a router which is not working,
>> we can see the following:
>>
>> Chain FORWARD (policy ACCEPT)
>>> target     prot opt source               destination
>>> BRIDGE-FIREWALL  all  --  anywhere             anywhere
>>>  PHYSDEV match --physdev-is-bridged
>>>
>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth1+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth4+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth3+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth7+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out bond3+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out bond0+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth6+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out bond1+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth5+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out bond2+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth0+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth2+ --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth1 --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth4 --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth3 --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth7 --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out bond3 --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth6 --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out bond1 --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth5 --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out bond2 --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth0 --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out eth2 --physdev-is-bridged
>>> ACCEPT     all  --  anywhere             anywhere            PHYSDEV
>>> match --physdev-out bond0 --physdev-is-bridged
>>> DROP       all  --  anywhere             anywhere
>>
>>
>> It seems the config is duplicated but without the pluses. I think these
>> are similar to wildcard entries?
>>
>> Cheers!
>>
>>
>>
>>
>>
>>
>>
>> On Fri, Mar 27, 2015 at 8:46 AM, Jayapal Reddy Uradi <
>> jayapalreddy.uradi@citrix.com> wrote:
>>
>>> Silly question but Is your xenserver configured bridge mode related
>>> settings correctly ?
>>>
>>> #xe-switch-network-backend bridge
>>> #echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
>>> #echo 1 > /proc/sys/net/bridge/bridge-nf-call-arptables
>>>
>>> Thanks,
>>> Jayapal
>>> On 27-Mar-2015, at 1:50 PM, cs user <acldstkusr@gmail.com<mailto:
>>> acldstkusr@gmail.com>> wrote:
>>>
>>> Hi Somesh,
>>>
>>> arping looks good, the correct mac address is displayed and we get a
>>> unicast reply from the ip address.
>>>
>>> Erik, tried restarting dnsmasq, all looks fine. VR is able to perform
>>> outgoing dns requests. There is nothing in syslog/dnsmasq logs that I can
>>> see. No egress rules are in place. The system vm's are able to perform
>>> dig's against google's dns, but not the virtual router. It seems it is
>>> being blocked at the xen level.
>>>
>>> We're seeing the below in the logs when restarting a network (either
>>> ticking clear config or not). This appears to be similar to :
>>>
>>> https://issues.apache.org/jira/browse/CLOUDSTACK-7605
>>>
>>> We are using basic zones, some have multiple pods, others don't. We see
>>> the
>>> same error in both. The routers come up though and go green, and dnsmasq
>>> is
>>> populated with the relevant info. DNS lookups work locally on the router,
>>> just not remotely. DHCP is working for new machines which get spun up.
>>>
>>> Is there a way to debug this? I've checked the logs on the router
>>> (cloud.log) can't see any errors in there.
>>>
>>> 2015-03-27 08:12:45,081 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (API-Job-Executor-16:ctx-0b0aa78a job-189235 ctx-77114e2e) Implementing
>>> the
>>> network Ntwk[9f5655bf-3101-45d9-83eb-d9061eadc2bb|Guest|47] elements and
>>> resources as a part of network restart
>>>
>>> 2015-03-27 08:12:45,096 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (API-Job-Executor-16:ctx-0b0aa78a job-189235 ctx-77114e2e) Asking
>>> SecurityGroupProvider to implemenet
>>> Ntwk[9f5655bf-3101-45d9-83eb-d9061eadc2bb|Guest|47]
>>>
>>> 2015-03-27 08:12:45,103 DEBUG [o.a.c.e.o.NetworkOrchestrator]
>>> (API-Job-Executor-16:ctx-0b0aa78a job-189235 ctx-77114e2e) Asking
>>> VirtualRouter to implemenet
>>> Ntwk[9f5655bf-3101-45d9-83eb-d9061eadc2bb|Guest|47]
>>>
>>> 2015-03-27 08:12:45,112 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>> (API-Job-Executor-16:ctx-0b0aa78a job-189235 ctx-77114e2e) Lock is
>>> acquired
>>> for network id 204 as a part of router startup in
>>>
>>> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>>> : Dest[Zone(8)-Pod(null)-Cluster(null)-Host(null)-Storage()]
>>>
>>> 2015-03-27 08:12:45,119 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>> (API-Job-Executor-16:ctx-0b0aa78a job-189235 ctx-77114e2e) Skipping VR
>>> deployment: Found a running or starting VR in Pod null id=8
>>>
>>> 2015-03-27 08:12:45,120 DEBUG
>>> [c.c.n.r.VirtualNetworkApplianceManagerImpl]
>>> (API-Job-Executor-16:ctx-0b0aa78a job-189235 ctx-77114e2e) Lock is
>>> released
>>> for network id 204 as a part of router startup in
>>>
>>> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
>>> : Dest[Zone(8)-Pod(null)-Cluster(null)-Host(null)-Storage()]
>>>
>>> 2015-03-27 08:12:45,123 WARN  [o.a.c.e.o.NetworkOrchestrator]
>>> (API-Job-Executor-16:ctx-0b0aa78a job-189235 ctx-77114e2e) Failed to
>>> implement network Ntwk[9f5655bf-3101-45d9-83eb-d9061eadc2bb|Guest|47]
>>> elements and resources as a part of network restart due to
>>>
>>> java.lang.NullPointerException
>>>
>>>        at
>>>
>>> com.cloud.network.element.VirtualRouterElement.getRouters(VirtualRouterElement.java:952)
>>>
>>>        at
>>>
>>> com.cloud.network.element.VirtualRouterElement.prepareAggregatedExecution(VirtualRouterElement.java:1099)
>>>
>>>        at
>>>
>>> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1090)
>>>
>>>        at
>>>
>>> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.restartNetwork(NetworkOrchestrator.java:2430)
>>>
>>>        at
>>>
>>> com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1892)
>>>
>>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>
>>>        at
>>>
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>
>>>        at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>>        at java.lang.reflect.Method.invoke(Method.java:601)
>>>
>>>        at
>>>
>>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>>>
>>>        at
>>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>>>
>>>        at
>>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>>>
>>>        at
>>>
>>> org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106)
>>>
>>>        at
>>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>>>
>>>        at
>>>
>>> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
>>>
>>>        at
>>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>>>
>>>        at
>>>
>>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>>>
>>>        at
>>>
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>>>
>>>        at
>>>
>>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>>>
>>>        at $Proxy156.restartNetwork(Unknown Source)
>>>
>>>        at
>>>
>>> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:95)
>>>
>>>        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
>>>
>>>        at
>>>
>>> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
>>>
>>>        at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
>>>
>>>        at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>>
>>>        at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>>
>>>        at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>>
>>>        at
>>>
>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>>
>>>        at
>>>
>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>>
>>>        at
>>>
>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
>>>
>>>        at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>
>>>        at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>>>
>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>>>
>>>        at
>>>
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>
>>>        at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>
>>>        at java.lang.Thread.run(Thread.java:722)
>>>
>>> 2015-03-27 08:12:45,125 WARN  [c.c.n.NetworkServiceImpl]
>>> (API-Job-Executor-16:ctx-0b0aa78a job-189235 ctx-77114e2e) Network id=204
>>> failed to restart.
>>>
>>> 2015-03-27 08:12:45,140 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (API-Job-Executor-16:ctx-0b0aa78a job-189235) Complete async job-189235,
>>> jobStatus: FAILED, resultCode: 530,
>>>
>>> result:org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed
>>> to restart network"}
>>>
>>> 2015-03-27 08:12:45,152 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (API-Job-Executor-16:ctx-0b0aa78a job-189235) Done executing
>>> org.apache.cloudstack.api.command.user.network.RestartNetworkCmd for
>>> job-189235
>>>
>>> 2015-03-27 08:12:45,158 INFO  [o.a.c.f.j.i.AsyncJobMonitor]
>>> (API-Job-Executor-16:ctx-0b0aa78a job-189235) Remove job-189235 from job
>>> monitoring
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Mar 26, 2015 at 7:38 PM, Somesh Naidu <Somesh.Naidu@citrix.com>
>>> wrote:
>>>
>>> You might want to do a arpping on the router's IP from one of the guests
>>> and see how many records are returned.
>>>
>>> Somesh
>>> CloudPlatform Escalations
>>> Citrix Systems, Inc.
>>>
>>> -----Original Message-----
>>> From: Erik Weber [mailto:terbolous@gmail.com]
>>> Sent: Thursday, March 26, 2015 12:54 PM
>>> To: users@cloudstack.apache.org
>>> Subject: Re: Virtual Routers not responding to dns requests
>>>
>>> I briefly remember having similar problems at some point, but do not
>>> recall
>>> details as version nor the solution.
>>>
>>> 1) Does it work if you restart dnsmasq on the VR?
>>> 2) is the VR able to do outgoing dns requests?
>>> 3) anything in syslog/dnsmasq logs?
>>> 4) any egress rules in place?
>>>
>>>
>>> Erik
>>>
>>> Den torsdag 26. mars 2015 skrev cs user <acldstkusr@gmail.com> følgende:
>>>
>>> Hi All,
>>>
>>> We have upgraded from 4.3 to 4.4.2.
>>>
>>> After some issues with starting the systemvm's, the virtual routers no
>>> longer responding to dns requests from the vm's which we start (or
>>> existing
>>> ones).
>>>
>>> We have disabled the firewall on the virtual routers and ran tcpdump on
>>> them but we can't see any inbound traffic on port 53 (udp or tcp). If we
>>> log onto the virtual routers and dig locally against eth0 and the alias
>>> on
>>> eth0 both of these return fine with the correct IP.
>>>
>>> This is using Xenserver 6.1 as the host.
>>>
>>> Has anyone come across this before? dhcp lookups appear to be working
>>> fine.
>>> Is there a firewall rule in place on the router vm's (other than
>>> iptables),
>>> similar to the security groups which are applied by Xen, which is
>>> preventing these requests from hitting the routers?
>>>
>>> Many thanks for any help.
>>>
>>>
>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message