incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcus Sorensen <shadow...@gmail.com>
Subject Re: SSVM Network Configuration Issue
Date Sat, 15 Dec 2012 06:19:33 GMT
Vlans in advanced/KVM should only be required for the guest networks. If I
create a bridge on physical eth0, name it 'br0', and a bridge on physical
eth1, naming it 'br1', and then set my management network to label 'br0',
my public network to 'br1', and my guest network to 'br1', it should use
the bridges you asked for when connecting the system VMs for the specified
traffic. I'd just leave the 'vlan' blank when specifying public and
pod(management) IPs. In this scenario, the only place you need to enter
vlans is on the guest, and it should create new tagged interfaces/bridges
on eth1(per your label of br1) as new guest networks are brought online.
This is how my dev VMs are usually set up.


On Thu, Dec 6, 2012 at 11:03 AM, John Burwell <jburwell@basho.com> wrote:

> Marcus,
>
> My question, more specifically, is are VLANs required to implement traffic
> labels?  Also, can traffic labels be configured in Basic networking mode or
> do I need to switch my configuration to Advanced?
>
> I am not disagreeing on the how DNS servers should be associated with
> interfaces nor do I think a network operator should be required to make any
> upstream router configuration changes.  I am simply saying that CloudStack
> should not make assumptions about the gateways that have been specified.
>  The behavior I experienced of CloudStack attempting to
> "correct" my configuration by injecting another route fails the rule of
> least surprise and is based on incomplete knowledge.  In my opinion,
> CloudStack (or any system of its ilk) should faithfully (or slavishly)
> realize the routes on the system VM as specified.  If the configuration is
> incorrect, networking will fail in an expected manner, and the operator can
> adjust their environment as necessary.   Otherwise, there is an upstream
> router configuration to which CloudStack has no visibility, but with which
> it is completely compatible.  Essentially, I am asking CloudStack to do
> less, assume I know what I am doing, and break in a manner consistent with
> other network applications.
>
> Thanks,
> -John
>
> On Dec 6, 2012, at 12:30 PM, Marcus Sorensen <shadowsor@gmail.com> wrote:
>
> > Traffic labels essentially tell the system which physical network to use.
> > So if you've allocated a vlan for a specific traffic type, it will first
> > look at the tag associated with that traffic type, figure out which
> > physical interface goes with that, and then create a tagged interface and
> > bridge also on that physical.
> >
> > I guess we'll just have to disagree, I think the current behavior makes
> > total sense.  To me, internal DNS should always use the management
> > interface, since it's internally facing. There's no sane way to do that
> > other than a static route on the system vm (it seems you're suggesting
> that
> > the network operator force something like this on the upstream router,
> > which seems really strange to require everyone to create static routes on
> > their public network to force specific IPs back into their internal
> > networks, so correct me if I have the wrong impression).  Cloudstack is
> > doing exactly what you tell it to. You told it that 10.0.3.2 should be
> > accessible via your internal network by setting it as your internal DNS.
> > The fact that a broken config doesn't work isn't CloudStack's fault.
> >
> > Note that internal DNS is just the default for the ssvm, public DNS is
> > still offered as a backup, so had you not said that 10.0.3.2 was
> available
> > on your internal network (perhaps offering a dummy internal DNS
> > address or 192.68.56.1),
> > lookups would fall back to public and everything would work as expected
> as
> > well.
> >
> > There is also a global config called 'use.external.dns', but in setting
> > this, restarting the management server, recreating system VMs, I don't
> see
> > a noticeable difference on any of this, so perhaps that would solve your
> > issue as well but it's either broken or doesn't do what I thought it
> would.
> >
> >
> > On Thu, Dec 6, 2012 at 8:39 AM, John Burwell <jburwell@basho.com> wrote:
> >
> >> Marcus,
> >>
> >> Are traffic labels independent of VLANs?  I ask because my current XCP
> >> network configuration is bridged, and I am not using Open vSwitch.
> >>
> >> I disagree on the routing issue.  CloudStack should do what's told
> because
> >> it does not have insight into or control of the configuration of the
> routes
> >> in the layers beneath it.  If CloudStack simply did as it was told, it
> >> would fail as expected in a typical networking environment while
> preserving
> >> the flexibility of configuration expected by a network engineer.
> >>
> >> Thanks,
> >> -John
> >>
> >> On Dec 6, 2012, at 10:35 AM, Marcus Sorensen <shadowsor@gmail.com>
> wrote:
> >>
> >>> I can't really tell you for xen, although it might be similar to KVM.
> >>> During setup I would set a traffic label matching the name of the
> bridge,
> >>> for example if my public interface were eth0 and the bridge I had set
> up
> >>> was br0, I'd go to the zone network settings, find public traffic, and
> >> set
> >>> a label on it of "br0". Maybe someone more familiar with the xen setup
> >> can
> >>> help.
> >>>
> >>> On the DNS, it makes sense from the perspective that the ssvm has
> access
> >> to
> >>> your internal networks, thus it uses your internal DNS. Its default
> >> gateway
> >>> is public. So if I have a DNS server on an internal network at
> >>> 10.30.20.10/24, and my management network on 192.168.10.0/24, this
> route
> >>> has to be set in order for the DNS server to be reachable. You would
> >> under
> >>> normal circumstances not want to use a DNS server on public net as your
> >>> internal DNS setting anyway, although I agree that the route insertion
> >>> should have a bit more sanity checking and not set a static route to
> your
> >>> default gateway.
> >>> On Dec 6, 2012 6:31 AM, "John Burwell" <jburwell@basho.com> wrote:
> >>>
> >>>> Marcus,
> >>>>
> >>>> I setup a small PowerDNS recursor on 192.168.56.15, configured the DNS
> >> for
> >>>> the management network to use it, and the route table in the SSVM is
> now
> >>>> correct.  However, this behavior does not seem correct.  At a minimum,
> >> it
> >>>> violates the rule of least surprise.  CloudStack shouldn't be adding
> >>>> gateways that are not configured.  Therefore, I have entered a
> >> defect[1] to
> >>>> remove the behavior.
> >>>>
> >>>> With the route table fixed, I am now experiencing a new problem.  The
> >>>> external NIC (10.0.3.0/24) on the SSVM is being connected to the
> >> internal
> >>>> NIC (192.168.56.0/24) on the host.  The host-only network
> >> (192.168.56.15)
> >>>> is configured on xenbr0 and the NAT network is configured on xenbr1.
> >> As a
> >>>> reference, the following is the contents of the
> /etc/network/interfaces
> >>>> file and ifconfig from devcloud host:
> >>>>
> >>>> root@zone1:/opt/cloudstack/apache-tomcat-6.0.32/bin# cat
> >>>> /etc/network/interfaces
> >>>> # The loopback network interface
> >>>> auto lo
> >>>> iface lo inet loopback
> >>>>
> >>>> auto eth0
> >>>> iface eth0 inet manual
> >>>>
> >>>> allow-hotplug eth1
> >>>> iface eth1 inet manual
> >>>>
> >>>> # The primary network interface
> >>>> auto xenbr0
> >>>> iface xenbr0 inet static
> >>>> address 192.168.56.15
> >>>> netmask 255.255.255.0
> >>>> network 192.168.56.0
> >>>> broadcast 192.168.56.255
> >>>> dns_nameserver 192.168.56.15
> >>>> bridge_ports eth0
> >>>>
> >>>> auto xenbr1
> >>>> iface xenbr1 inet dhcp
> >>>> bridge_ports eth1
> >>>> dns_nameserver 8.8.8.8 8.8.4.4
> >>>> post-up route add default gw 10.0.3.2
> >>>>
> >>>> root@zone1:/opt/cloudstack/apache-tomcat-6.0.32/bin# ifconfig
> >>>> eth0      Link encap:Ethernet  HWaddr 08:00:27:7e:74:9c
> >>>>         inet6 addr: fe80::a00:27ff:fe7e:749c/64 Scope:Link
> >>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>>>         RX packets:777 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:188 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:1000
> >>>>         RX bytes:109977 (109.9 KB)  TX bytes:11900 (11.9 KB)
> >>>>
> >>>> eth1      Link encap:Ethernet  HWaddr 08:00:27:df:00:00
> >>>>         inet6 addr: fe80::a00:27ff:fedf:0/64 Scope:Link
> >>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>>>         RX packets:4129 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:3910 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:1000
> >>>>         RX bytes:478719 (478.7 KB)  TX bytes:2542459 (2.5 MB)
> >>>>
> >>>> lo        Link encap:Local Loopback
> >>>>         inet addr:127.0.0.1  Mask:255.0.0.0
> >>>>         inet6 addr: ::1/128 Scope:Host
> >>>>         UP LOOPBACK RUNNING  MTU:16436  Metric:1
> >>>>         RX packets:360285 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:360285 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:0
> >>>>         RX bytes:169128181 (169.1 MB)  TX bytes:169128181 (169.1 MB)
> >>>>
> >>>> vif1.0    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff
> >>>>         inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
> >>>>         UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
> >>>>         RX packets:6 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:152 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:32
> >>>>         RX bytes:292 (292.0 B)  TX bytes:9252 (9.2 KB)
> >>>>
> >>>> vif1.1    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff
> >>>>         inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
> >>>>         UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
> >>>>         RX packets:566 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:1405 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:32
> >>>>         RX bytes:44227 (44.2 KB)  TX bytes:173995 (173.9 KB)
> >>>>
> >>>> vif1.2    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff
> >>>>         inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
> >>>>         UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
> >>>>         RX packets:3 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:838 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:32
> >>>>         RX bytes:84 (84.0 B)  TX bytes:111361 (111.3 KB)
> >>>>
> >>>> vif4.0    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff
> >>>>         inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
> >>>>         UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
> >>>>         RX packets:64 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:197 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:32
> >>>>         RX bytes:10276 (10.2 KB)  TX bytes:18453 (18.4 KB)
> >>>>
> >>>> vif4.1    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff
> >>>>         inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
> >>>>         UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
> >>>>         RX packets:2051 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:2446 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:32
> >>>>         RX bytes:233914 (233.9 KB)  TX bytes:364243 (364.2 KB)
> >>>>
> >>>> vif4.2    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff
> >>>>         inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
> >>>>         UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
> >>>>         RX packets:3 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:582 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:32
> >>>>         RX bytes:84 (84.0 B)  TX bytes:74700 (74.7 KB)
> >>>>
> >>>> vif4.3    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff
> >>>>         inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
> >>>>         UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
> >>>>         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:585 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:32
> >>>>         RX bytes:0 (0.0 B)  TX bytes:74826 (74.8 KB)
> >>>>
> >>>> xapi0     Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff
> >>>>         inet addr:169.254.0.1  Bcast:169.254.255.255  Mask:255.255.0.0
> >>>>         inet6 addr: fe80::c870:1aff:fec2:22b/64 Scope:Link
> >>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>>>         RX packets:568 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:1132 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:0
> >>>>         RX bytes:76284 (76.2 KB)  TX bytes:109085 (109.0 KB)
> >>>>
> >>>> xenbr0    Link encap:Ethernet  HWaddr 08:00:27:7e:74:9c
> >>>>         inet addr:192.168.56.15  Bcast:192.168.56.255
> >> Mask:255.255.255.0
> >>>>         inet6 addr: fe80::a00:27ff:fe7e:749c/64 Scope:Link
> >>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>>>         RX packets:4162 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:3281 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:0
> >>>>         RX bytes:469199 (469.1 KB)  TX bytes:485688 (485.6 KB)
> >>>>
> >>>> xenbr1    Link encap:Ethernet  HWaddr 08:00:27:df:00:00
> >>>>         inet addr:10.0.3.15  Bcast:10.0.3.255  Mask:255.255.255.0
> >>>>         inet6 addr: fe80::a00:27ff:fedf:0/64 Scope:Link
> >>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>>>         RX packets:4129 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:3114 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:0
> >>>>         RX bytes:404327 (404.3 KB)  TX bytes:2501443 (2.5 MB)
> >>>>
> >>>> These physical NICs on the host translate to the following Xen PIFs:
> >>>>
> >>>> root@zone1:/opt/cloudstack/apache-tomcat-6.0.32/bin# xe pif-list
> >>>> uuid ( RO)                  : 207413c9-5058-7a40-6c96-2dab21057f30
> >>>>               device ( RO): eth1
> >>>>   currently-attached ( RO): true
> >>>>                 VLAN ( RO): -1
> >>>>         network-uuid ( RO): 1679ddb1-5a21-b827-ab07-c16275d5ce72
> >>>>
> >>>>
> >>>> uuid ( RO)                  : c0274787-e768-506f-3191-f0ac17b0c72b
> >>>>               device ( RO): eth0
> >>>>   currently-attached ( RO): true
> >>>>                 VLAN ( RO): -1
> >>>>         network-uuid ( RO): 8ee927b1-a35d-ac10-4471-d7a6a475839a
> >>>>
> >>>> The following is the ifconfig from the SSVM:
> >>>>
> >>>> root@s-5-TEST:~# ifconfig
> >>>> eth0      Link encap:Ethernet  HWaddr 0e:00:a9:fe:03:8b
> >>>>         inet addr:169.254.3.139  Bcast:169.254.255.255
> >> Mask:255.255.0.0
> >>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>>>         RX packets:235 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:92 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:1000
> >>>>         RX bytes:21966 (21.4 KiB)  TX bytes:16404 (16.0 KiB)
> >>>>         Interrupt:8
> >>>>
> >>>> eth1      Link encap:Ethernet  HWaddr 06:bc:62:00:00:05
> >>>>         inet addr:192.168.56.104  Bcast:192.168.56.255
> >>>> Mask:255.255.255.0
> >>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>>>         RX packets:2532 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:2127 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:1000
> >>>>         RX bytes:341242 (333.2 KiB)  TX bytes:272183 (265.8 KiB)
> >>>>         Interrupt:10
> >>>>
> >>>> eth2      Link encap:Ethernet  HWaddr 06:12:72:00:00:37
> >>>>         inet addr:10.0.3.204  Bcast:10.0.3.255  Mask:255.255.255.0
> >>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>>>         RX packets:600 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:1000
> >>>>         RX bytes:68648 (67.0 KiB)  TX bytes:126 (126.0 B)
> >>>>         Interrupt:11
> >>>>
> >>>> eth3      Link encap:Ethernet  HWaddr 06:25:e2:00:00:15
> >>>>         inet addr:192.168.56.120  Bcast:192.168.56.255
> >>>> Mask:255.255.255.0
> >>>>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>>>         RX packets:603 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:1000
> >>>>         RX bytes:68732 (67.1 KiB)  TX bytes:0 (0.0 B)
> >>>>         Interrupt:12
> >>>>
> >>>> lo        Link encap:Local Loopback
> >>>>         inet addr:127.0.0.1  Mask:255.0.0.0
> >>>>         UP LOOPBACK RUNNING  MTU:16436  Metric:1
> >>>>         RX packets:61 errors:0 dropped:0 overruns:0 frame:0
> >>>>         TX packets:61 errors:0 dropped:0 overruns:0 carrier:0
> >>>>         collisions:0 txqueuelen:0
> >>>>         RX bytes:5300 (5.1 KiB)  TX bytes:5300 (5.1 KiB)
> >>>>
> >>>> Finally, the following are the vif params for the eth2 device on the
> >> SSVM
> >>>> depicting its connection to eth0 instead of eth1:
> >>>>
> >>>> root@zone1:/opt/cloudstack/apache-tomcat-6.0.32/bin# !1243
> >>>> xe vif-param-list uuid=be44bb30-5700-b461-760e-10fe93079210
> >>>> uuid ( RO)                        :
> be44bb30-5700-b461-760e-10fe93079210
> >>>>                    vm-uuid ( RO): 7958d91f-e52d-a25d-718c-7f831ae701d7
> >>>>              vm-name-label ( RO): s-5-TEST
> >>>>         allowed-operations (SRO): attach; unplug_force; unplug
> >>>>         current-operations (SRO):
> >>>>                     device ( RO): 2
> >>>>                        MAC ( RO): 06:12:72:00:00:37
> >>>>          MAC-autogenerated ( RO): false
> >>>>                        MTU ( RO): 1500
> >>>>         currently-attached ( RO): true
> >>>>         qos_algorithm_type ( RW): ratelimit
> >>>>       qos_algorithm_params (MRW): kbps: 25600
> >>>>   qos_supported_algorithms (SRO):
> >>>>               other-config (MRW): nicira-iface-id:
> >>>> 3d68b9f8-98d1-4ac7-92d8-fb57cb8b0adc; nicira-vm-id:
> >>>> 7958d91f-e52d-a25d-718c-7f831ae701d7
> >>>>               network-uuid ( RO): 8ee927b1-a35d-ac10-4471-d7a6a475839a
> >>>>         network-name-label ( RO): Pool-wide network associated with
> >> eth0
> >>>>                io_read_kbs ( RO): 0.007
> >>>>               io_write_kbs ( RO): 0.000
> >>>>
> >>>> How do I configure CloudStack such that the guest network NIC on the
> VM
> >>>> will be connected to correct physical NIC?
> >>>>
> >>>> Thanks for your help,
> >>>> -John
> >>>>
> >>>>
> >>>> [1]: https://issues.apache.org/jira/browse/CLOUDSTACK-590
> >>>>
> >>>> On Dec 5, 2012, at 2:47 PM, Marcus Sorensen <shadowsor@gmail.com>
> >> wrote:
> >>>>
> >>>>> Yes, see your cmdline. internaldns1=10.0.3.2, so it is forcing the
> use
> >> of
> >>>>> management network to route to 10.0.3.2 for DNS. that's where the
> route
> >>>> is
> >>>>> coming from. you will want to use something on your management net
> for
> >>>>> internal DNS, or something other than that router.
> >>>>>
> >>>>>
> >>>>> On Wed, Dec 5, 2012 at 11:59 AM, John Burwell <jburwell@basho.com>
> >>>> wrote:
> >>>>>
> >>>>>> Anthony,
> >>>>>>
> >>>>>> I apologize for forgetting to response to the part of your answer
> the
> >>>>>> first part of the question.  I had set the management.network.cidr
> and
> >>>> host
> >>>>>> global settings to 192.168.0.0/24 and 192.168.56.18 respectively.
> >>>> Please
> >>>>>> see the zone1.devcloud.cfg Marvin configuration attached to
my
> >> original
> >>>>>> email for the actual setting, as well as, the network configurations
> >>>> used
> >>>>>> when this problem occurs.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> -John
> >>>>>>
> >>>>>> On Dec 5, 2012, at 12:46 PM, Anthony Xu <Xuefei.Xu@citrix.com>
> wrote:
> >>>>>>
> >>>>>>> Hi join,
> >>>>>>>
> >>>>>>> Try following,
> >>>>>>>
> >>>>>>> Set global configuration management.network.cidr to your
management
> >>>>>> server CIDR, if this configuration is not available in UI, you
can
> >>>> change
> >>>>>> it in DB directly.
> >>>>>>>
> >>>>>>> Restart management,
> >>>>>>> Stop/Start SSVM and CPVM.
> >>>>>>>
> >>>>>>>
> >>>>>>> And could you post "cat /proc/cmdline" in SSVM?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Anthony
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: John Burwell [mailto:jburwell@basho.com]
> >>>>>>>> Sent: Wednesday, December 05, 2012 9:11 AM
> >>>>>>>> To: cloudstack-dev@incubator.apache.org
> >>>>>>>> Subject: Re: SSVM Network Configuration Issue
> >>>>>>>>
> >>>>>>>> All,
> >>>>>>>>
> >>>>>>>> I was wondering if anyone else is experiencing this
problem when
> >> using
> >>>>>>>> secondary storage on a devcloud-style VM with a host-only
and NAT
> >>>>>>>> adapter.  One aspect of this issue that seems interesting
is that
> >>>>>>>> following route table from the SSVM:
> >>>>>>>>
> >>>>>>>> root@s-5-TEST:~# route
> >>>>>>>> Kernel IP routing table
> >>>>>>>> Destination     Gateway         Genmask         Flags
Metric Ref
> >>>> Use
> >>>>>>>> Iface
> >>>>>>>> 10.0.3.2        192.168.56.1    255.255.255.255 UGH
  0      0
> >>>> 0
> >>>>>>>> eth1
> >>>>>>>> 10.0.3.0        *               255.255.255.0   U  
  0      0
> >>>> 0
> >>>>>>>> eth2
> >>>>>>>> 192.168.56.0    *               255.255.255.0   U  
  0      0
> >>>> 0
> >>>>>>>> eth1
> >>>>>>>> 192.168.56.0    *               255.255.255.0   U  
  0      0
> >>>> 0
> >>>>>>>> eth3
> >>>>>>>> link-local      *               255.255.0.0     U  
  0      0
> >>>> 0
> >>>>>>>> eth0
> >>>>>>>> default         10.0.3.2        0.0.0.0         UG 
  0      0
> >>>> 0
> >>>>>>>> eth2
> >>>>>>>>
> >>>>>>>> In particular, the gateways for the management and guest
networks
> do
> >>>>>>>> not match to the configuration provided to the management
server
> >> (i.e.
> >>>>>>>> 10.0.3.2 is the gateway for the 10.0.3.0/24 network
and
> >> 192.168.56.1
> >>>> is
> >>>>>>>> the gateway for the 192.168.56.0/24 network).  With
this
> >>>> configuration,
> >>>>>>>> the SSVM has a socket connection to the management server,
but is
> in
> >>>>>>>> alert state.  Finally, when I remove the host-only NIC
and use
> only
> >> a
> >>>>>>>> NAT adapter the SSVM's networking works as expecting
leading me to
> >>>>>>>> believe that the segregated network configuration is
at the root
> of
> >>>> the
> >>>>>>>> problem.
> >>>>>>>>
> >>>>>>>> Until I can get the networking on the SSVM configured,
I am unable
> >> to
> >>>>>>>> complete the testing of the S3-backed Secondary Storage
> enhancement.
> >>>>>>>>
> >>>>>>>> Thank you for your help,
> >>>>>>>> -John
> >>>>>>>>
> >>>>>>>> On Dec 3, 2012, at 4:46 PM, John Burwell <jburwell@basho.com>
> >> wrote:
> >>>>>>>>
> >>>>>>>>> All,
> >>>>>>>>>
> >>>>>>>>> I am setting up a multi-zone devcloud configuration
on VirtualBox
> >>>>>>>> 4.2.4 using the Ubuntu 12.04.1 and Xen 4.1.  I have
configured the
> >>>> base
> >>>>>>>> management server VM (zone1) to serve as both the zone1,
as well
> as,
> >>>>>>>> the management server (running MySql) with eth0 as a
host-only
> >> adapter
> >>>>>>>> and a static IP of 192.168.56.15 and eth1 as a NAT adapter
(see
> the
> >>>>>>>> attached zone1-interfaces file for the exact network
configuration
> >> on
> >>>>>>>> the VM).  The management and guest networks are configured
as
> >> follows:
> >>>>>>>>>
> >>>>>>>>> Zone 1
> >>>>>>>>> Management: 192.168.56.100-149 gw 192.168.56.1 dns
10.0.3.2 (?)
> >>>>>>>>> Guest: 10.0.3.200-10.0.3.220 gw 10.0.3.2 dns 8.8.8.8
> >>>>>>>>> Zone 2
> >>>>>>>>> Management: 192.168.56.150-200 gw 192.68.56.1 dns
10.0.3.2 (?)
> >>>>>>>>> Guest: 10.0.3.221-240 gw 10.0.3.2 dns 8.8.8.8
> >>>>>>>>>
> >>>>>>>>> The management server deploys and starts without
error.  I then
> >>>>>>>> populate the configuration it using the attached Marvin
> >> configuration
> >>>>>>>> file (zone1.devcloud.cfg) and restart the management
server in
> order
> >>>> to
> >>>>>>>> allow the global configuration option changes to take
effect.
> >>>>>>>> Following the restart, the CPVM and SSVM start without
error.
> >>>>>>>> Unfortunately, they drop into alert status, and the
SSVM is unable
> >> to
> >>>>>>>> connect outbound through the guest network (very important
for my
> >>>> tests
> >>>>>>>> because I am testing S3-backed secondary storage).
> >>>>>>>>>
> >>>>>>>>> From the diagnostic checks I have performed on the
management
> >> server
> >>>>>>>> and the SSVM, it appears that the daemon on the SSVM
is connecting
> >>>> back
> >>>>>>>> to the management server.  I have attached a set of
diagnostic
> >>>>>>>> information from the management server
> >> (mgmtsvr-zone1-diagnostics.log)
> >>>>>>>> and SSVM server (ssvm-zone1-diagnostics.log) that includes
the
> >> results
> >>>>>>>> of ifconfig, route, netstat and ping checks, as well
as, other
> >>>>>>>> information (e.g. the contents of /var/cache/cloud/cmdline
on the
> >>>> SSVM).
> >>>>>>>> Finally, I have attached the vmops log from the management
server
> >>>>>>>> (vmops-zone1.log).
> >>>>>>>>>
> >>>>>>>>> What changes need to be made to management server
configuration
> in
> >>>>>>>> order to start up an SSVM that can communicate with
the secondary
> >>>>>>>> storage NFS volumes, management server, and connect
to hosts on
> the
> >>>>>>>> Internet?
> >>>>>>>>>
> >>>>>>>>> Thanks for your help,
> >>>>>>>>> -John
> >>>>>>>>>
> >>>>>>>>> <ssvm-zone1-diagnostics.log>
> >>>>>>>>> <vmops-zone1.tar.gz>
> >>>>>>>>> <mgmtsvr-zone1-diagnostics.log>
> >>>>>>>>> <zone1-interfaces>
> >>>>>>>>> <zone1.devcloud.cfg>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message