cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daan Hoogland <daan.hoogl...@gmail.com>
Subject Re: VPC's VR missing public NIC eth1
Date Wed, 28 May 2014 19:34:40 GMT
Andrija,

nevertheless it sounds familiar. I will be back in the office on
monday and ask around.

On Wed, May 28, 2014 at 9:23 PM, Andrija Panic <andrija.panic@gmail.com> wrote:
> Hi Daan,
>
> I don't think this is my issue, at least I don't make use of private
> gateway - this is just simple as:   create new VPC from scratch - Public IP
> is not assigned to VR eth1 interface inside VR...
>
> I have filed the bug: https://issues.apache.org/jira/browse/CLOUDSTACK-6801
>
> This same thing happened previously to Andrei Mikhailovsky:
> http://mail-archives.apache.org/mod_mbox/cloudstack-users/201405.mbox/%3C33347835.250.1399336340785.JavaMail.andrei@tuchka%3Eand
> it is not resolved
>
> Thanks,
>
> Andrija
>
>
> On 28 May 2014 21:01, Daan Hoogland <daan.hoogland@gmail.com> wrote:
>
>> Andrija,
>>
>> this sound like something we seen as well.
>> can you check if this is it :
>> https://issues.apache.org/jira/browse/CLOUDSTACK-6485
>>
>> thanks,
>> Daan
>>
>> On Wed, May 28, 2014 at 3:30 PM, Andrija Panic <andrija.panic@gmail.com>
>> wrote:
>> > Hi there,
>> >
>> > I'm having big time problems with Public IP missing from VPC VR's eth1,
>> > after upgrade to ACS 4.3.1 - did not found this filed as bug so far...and
>> > it worked all fine on ACS 4.2.1.
>> >
>> > No help so far from user mailing list...
>> >
>> > Below is a detailed explanation, and logs from inside VR, and from
>> > management (all fine with management logs...)
>> >
>> > If anybody can help,  I would very much appriciate this, since now I have
>> > bunch fo VPC unoperational...
>> >
>> > Thanks
>> >
>> > ---------- Forwarded message ----------
>> > From: Andrija Panic <andrija.panic@gmail.com>
>> > Date: 28 May 2014 14:50
>> > Subject: Re: VPC's VR missing public NIC eth1
>> > To: users@cloudstack.apache.org
>> >
>> >
>> > and as I said eth1 is present:
>> >
>> > root@r-794-VM:~# cat /proc/net/dev
>> > Inter-|   Receive                                                |
>>  Transmit
>> >  face |bytes    packets errs drop fifo frame compressed multicast|bytes
>> >  packets errs drop fifo colls carrier compressed
>> >   eth3:   11484     131    0    0    0     0          0         0
>>  11590
>> >     131    0    0    0     0       0          0
>> >     lo:     214       2    0    0    0     0          0         0
>>  214
>> >       2    0    0    0     0       0          0
>> >   eth2:   32970     544    0    0    0     0          0         0
>> 2084
>> >      24    0    0    0     0       0          0
>> >   eth1:       0       0    0    0    0     0          0         0
>>  0
>> >       0    0    0    0     0       0          0
>> >   eth0:  150207    1319    0    0    0     0          0         0
>> 264232
>> >    1180    0    0    0     0       0          0
>> >
>> >
>> > On 28 May 2014 14:47, Andrija Panic <andrija.panic@gmail.com> wrote:
>> >
>> >> Also, from /var/log/messages/ inside VR:
>> >>
>> >> This is a major show stopper - all our VPCs are unusable complete.
>> >> Anybody... ?
>> >>
>> >> May 28 12:37:33 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 0 seconds
>> >> May 28 12:37:34 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 1 seconds
>> >> May 28 12:37:35 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 2 seconds
>> >> May 28 12:37:36 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 3 seconds
>> >> May 28 12:37:37 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 4 seconds
>> >> May 28 12:37:38 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 5 seconds
>> >> May 28 12:37:39 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 6 seconds
>> >> May 28 12:37:40 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 7 seconds
>> >> May 28 12:37:41 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 8 seconds
>> >> May 28 12:37:42 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 9 seconds
>> >> May 28 12:37:43 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 10 seconds
>> >> May 28 12:37:44 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 11 seconds
>> >> May 28 12:37:45 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 12 seconds
>> >> May 28 12:37:46 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 13 seconds
>> >> May 28 12:37:47 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 14 seconds
>> >> May 28 12:37:48 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 15 seconds
>> >> May 28 12:37:49 r-794-VM cloud: vpc_ipassoc.sh:Waiting for interface
>> >> ethnull to appear, 16 seconds
>> >> May 28 12:37:50 r-794-VM cloud: vpc_ipassoc.sh:interface ethnull never
>> >> appeared
>> >> May 28 12:37:50 r-794-VM cloud: vpc_ipassoc.sh:Adding ip 46.232.180.246
>> on
>> >> interface ethnull
>> >> May 28 12:37:50 r-794-VM cloud: vpc_ipassoc.sh:Add routing
>> 46.232.180.246
>> >> on interface ethnull
>> >> May 28 12:37:50 r-794-VM cloud: vpc_privateGateway.sh:Added SourceNAT
>> >> 46.232.180.246 on interface ethnull
>> >> May 28 12:37:50 r-794-VM cloud: vpc_snat.sh:Added SourceNAT
>> 46.232.180.246
>> >> on interface eth1
>> >>
>> >>
>> >>
>> >>
>> >> On 28 May 2014 12:59, Andrija Panic <andrija.panic@gmail.com> wrote:
>> >>
>> >>> Defined eth1 manually inside /etc/network/interfaces inside VPC's VR.
>> >>> iface  eth1 inet static
>> >>>   address 46.232.180.246
>> >>>   netmask 255.255.255.0
>> >>>
>> >>> ifup eth1
>> >>> ip route add default via 46.232.180.1
>> >>>
>> >>> so now VR works fine (have access to internet)
>> >>>
>> >>> But again, adding new IP to VR, and enabling static NAT is failing...
>> >>> That is, geting new IP works fine (just associated with account)
>> >>> But enabling static NAT fails, due to "resource unavailable"
>> >>>
>> >>> Here are management logs:
>> >>> 2014-05-28 12:57:00,716 WARN  [c.c.n.r.RulesManagerImpl]
>> >>> (catalina-exec-22:ctx-537ac57b ctx-8c44c786) Failed to create static
>> nat
>> >>> rule due to
>> >>> com.cloud.exception.ResourceUnavailableException: Resource
>> [DataCenter:1]
>> >>> is unreachable: Unable to apply static nat rules on router
>> >>>         at
>> >>>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3915)
>> >>>         at
>> >>>
>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyStaticNats(VirtualNetworkApplianceManagerImpl.java:3963)
>> >>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >>>         at
>> >>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> >>>         at
>> >>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >>>         at java.lang.reflect.Method.invoke(Method.java:622)
>> >>>         at
>> >>>
>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>> >>>         at
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>> >>>         at
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>> >>>         at
>> >>>
>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>> >>>         at
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>> >>>         at
>> >>>
>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>> >>>         at com.sun.proxy.$Proxy240.applyStaticNats(Unknown Source)
>> >>>         at
>> >>>
>> com.cloud.network.element.VirtualRouterElement.applyStaticNats(VirtualRouterElement.java:650)
>> >>>         at
>> >>>
>> com.cloud.network.IpAddressManagerImpl.applyStaticNats(IpAddressManagerImpl.java:1762)
>> >>>         at
>> >>>
>> com.cloud.network.rules.RulesManagerImpl.applyStaticNatForIp(RulesManagerImpl.java:1324)
>> >>>         at
>> >>>
>> com.cloud.network.rules.RulesManagerImpl.enableStaticNat(RulesManagerImpl.java:602)
>> >>>         at
>> >>>
>> com.cloud.network.rules.RulesManagerImpl.enableStaticNat(RulesManagerImpl.java:446)
>> >>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >>>         at
>> >>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> >>>         at
>> >>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >>>         at java.lang.reflect.Method.invoke(Method.java:622)
>> >>>         at
>> >>>
>> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>> >>>         at
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>> >>>         at
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>> >>>         at
>> >>>
>> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50)
>> >>>         at
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>> >>>         at
>> >>>
>> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>> >>>         at
>> >>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>> >>>         at
>> >>>
>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>> >>>         at com.sun.proxy.$Proxy88.enableStaticNat(Unknown Source)
>> >>>         at
>> >>>
>> org.apache.cloudstack.api.command.user.nat.EnableStaticNatCmd.execute(EnableStaticNatCmd.java:129)
>> >>>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
>> >>>         at com.cloud.api.ApiServer.queueCommand(ApiServer.java:531)
>> >>>         at com.cloud.api.ApiServer.handleRequest(ApiServer.java:374)
>> >>>         at
>> >>> com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:323)
>> >>>         at com.cloud.api.ApiServlet.access$000(ApiServlet.java:53)
>> >>>         at com.cloud.api.ApiServlet$1.run(ApiServlet.java:115)
>> >>>         at
>> >>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>> >>>         at
>> >>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>> >>>         at
>> >>>
>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>> >>>         at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:112)
>> >>>         at com.cloud.api.ApiServlet.doGet(ApiServlet.java:74)
>> >>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>> >>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>> >>>         at
>> >>>
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>> >>>         at
>> >>>
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>> >>>         at
>> >>>
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>> >>>         at
>> >>>
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>> >>>         at
>> >>>
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>> >>>         at
>> >>>
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>> >>>         at
>> >>>
>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
>> >>>         at
>> >>>
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>> >>>         at
>> >>>
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>> >>>         at
>> >>>
>> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
>> >>>         at
>> >>>
>> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
>> >>>         at
>> >>>
>> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
>> >>>         at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
>> >>>         at
>> >>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> >>>         at java.lang.Thread.run(Thread.java:701)
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On 28 May 2014 00:58, Andrija Panic <andrija.panic@gmail.com>
wrote:
>> >>>
>> >>>> Hi Jayapal,
>> >>>>
>> >>>> eth1 seems present (lspci and virsh comfirmed), but not started
inside
>> >>>> VPC's VR - (VR used for Shared Network is fine)...
>> >>>> I could confirm by virsh that is is plugged inside appropriate bridge
>> >>>> breth1-500 (check management logs...)
>> >>>>
>> >>>> management log while createing new VPC (VR) -
>> >>>> http://pastebin.com/s77nu5Ei
>> >>>> The public IP is there, so command is fine for creating it I guess...
>> >>>>
>> >>>> VR's /var/log/cloud.log after rebooting VR from CS GUI
>> >>>> Tue May 27 22:46:58 UTC 2014 Executing cloud-early-config
>> >>>> Tue May 27 22:46:58 UTC 2014 Detected that we are running inside
kvm
>> >>>> guest
>> >>>> Tue May 27 22:46:59 UTC 2014 Patching  cloud service
>> >>>> Tue May 27 22:47:00 UTC 2014 Updating log4j-cloud.xml
>> >>>> Tue May 27 22:47:00 UTC 2014 Setting up VPC virtual router system
vm
>> >>>> Tue May 27 22:47:00 UTC 2014 checking that eth0 has IP
>> >>>> Tue May 27 22:47:00 UTC 2014 Setting up apache web server for VPC
>> >>>> Tue May 27 22:47:00 UTC 2014 Enable service dnsmasq = 1
>> >>>> Tue May 27 22:47:00 UTC 2014 Enable service haproxy = 1
>> >>>> Tue May 27 22:47:00 UTC 2014 Processors = 1  Enable service  = 0
>> >>>> Tue May 27 22:47:00 UTC 2014 Enable service cloud = 0
>> >>>> Tue May 27 22:47:00 UTC 2014 cloud: disable rp_filter
>> >>>> Tue May 27 22:47:00 UTC 2014 disable rpfilter
>> >>>> Tue May 27 22:47:00 UTC 2014 cloud: enable_fwding = 1
>> >>>> Tue May 27 22:47:00 UTC 2014 enable_fwding = 1
>> >>>>
>> >>>> ifconfig (no eth1 shown)
>> >>>>
>> >>>> eth0      Link encap:Ethernet  HWaddr 0e:00:a9:fe:03:5c
>> >>>>           inet addr:169.254.3.92  Bcast:169.254.255.255
>>  Mask:255.255.0.0
>> >>>>
>> >>>> eth2      Link encap:Ethernet  HWaddr 02:00:7d:92:00:10
>> >>>>           inet addr:10.0.1.1  Bcast:10.0.1.255  Mask:255.255.255.0
>> >>>>
>> >>>> eth3      Link encap:Ethernet  HWaddr 02:00:78:e9:00:05
>> >>>>           inet addr:10.0.3.1  Bcast:10.0.3.255  Mask:255.255.255.0
>> >>>>
>> >>>> lo        Link encap:Local Loopback
>> >>>>           inet addr:127.0.0.1  Mask:255.0.0.0
>> >>>>
>> >>>>
>> >>>> cat /etc/network/interfaces
>> >>>> auto lo eth0
>> >>>> iface lo inet loopback
>> >>>> iface  eth0 inet static
>> >>>>   address 169.254.3.92
>> >>>>   netmask 255.255.0.0
>> >>>>
>> >>>> lspci - shows 4 ehternet addapters
>> >>>> ethtool eth1 = no link detected
>> >>>> virsh - confirmed that eth1 is plugged to correct bridge (breth1-500)
>> as
>> >>>> indicated by management logs, and shows good MAC address as shown
in
>> >>>> managemetn log on pastebin..
>> >>>>
>> >>>> This is completely makeing VPCs unusable...
>> >>>> :(
>> >>>>
>> >>>> Cheers
>> >>>>
>> >>>>
>> >>>> On 27 May 2014 16:36, Jayapal Reddy Uradi <
>> jayapalreddy.uradi@citrix.com
>> >>>> > wrote:
>> >>>>
>> >>>>> Hi,
>> >>>>> Can you please share management server and router logs in
>> pastebin.comto understand the issue ?
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Jayapal
>> >>>>>
>> >>>>> On 27-May-2014, at 6:21 PM, Andrija Panic <andrija.panic@gmail.com>
>> >>>>>  wrote:
>> >>>>>
>> >>>>> > Hi,
>> >>>>> >
>> >>>>> > after the upgrade to ACS 4.3 (from 4.2.1) existing VRs
for VPC lost
>> >>>>> their
>> >>>>> > eth1 which is public NIC. VR got eth0(control nic) and
eth2 and
>> eth3
>> >>>>> (bith
>> >>>>> > belonging to Tiers). From CS GUI, it is reported that the
VR has
>> eth1
>> >>>>> with
>> >>>>> > Public network attached, but from inside (ssh to VR) there
is no
>> eth1
>> >>>>> with
>> >>>>> > public IP...
>> >>>>> >
>> >>>>> > Even after destroying those VR, they are recreated again,
but
>> without
>> >>>>> eth1.
>> >>>>> >
>> >>>>> > Anybody experienced same situtation ?
>> >>>>> >
>> >>>>> > Thanks,
>> >>>>> >
>> >>>>> > --
>> >>>>> >
>> >>>>> > Andrija Panić
>> >>>>> > --------------------------------------
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>>
>> >>>>
>> >>>>
>>
>>
>>
>> --
>> Daan
>>
>
>
>
> --
>
> Andrija Panić
> --------------------------------------
>   http://admintweets.com
> --------------------------------------



-- 
Daan

Mime
View raw message