vcl-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy Kurth <andy_ku...@ncsu.edu>
Subject Re: Additional VM Networks in Virtual Host profile
Date Thu, 02 Aug 2012 17:19:30 GMT
To clarify the image.project functionality...  This was added a while
ago as a somewhat easy hack to automatically add additional interfaces
to VMs on an image by image basis.  It compares the image.project
value with all of the network names defined on the VM host and adds
additional interfaces to the VM if a match is detected.  We mainly use
it to add a 3rd interface to our images which need access to a special
HPC network.

As you saw in the code, it first checks if project is simply 'vcl':
if ($image_project !~ /^vcl$/i

If it is 'vcl', no additional interfaces are added.  If it is not
'vcl', additional checks are done.  The code retrieves the list of all
of the networks on the VM host and compares them to the image.project
name.  Additional interfaces are added for any overlapping
project-to-network names.

Since image.project is an enum, you can edit the column definition to
add additional values such as 'vclhpc'.  We also define an additional
network on our VM hosts which we want the HPC images to run on called
'HPC'.  A match is detected when the code compares the project name
'vclhpc' to the 'HPC' network:
if ($network_name =~ /$image_project/i || $image_project =~ /$network_name/i) {

You can define as many additional interfaces as you want.  For example:
VM host has 3 extra networks added: vlan1, vlan2, vlan3

If image.project is 'vlan1-vlan2' 2 additional interfaces would be
added: vlan1 and vlan2.  If you had another image with project set to
'vlan2-vlan3', it would receive 2 additional interfaces on vlan2 and
vlan3.

-Andy

On Thu, Aug 2, 2012 at 12:30 PM, Dmitri Chebotarov <dchebota@gmu.edu> wrote:
> Hi
>
> Mani, Open vSwitch functionality will be nice addition to VCL, no arguments here.
>
> In my case I'm looking for a solution to address following requests: (a) is to have isolated
networking within VCL reservations which are based on the same image, and (b) is to allow
access to existing University networks.
> Whoever designed VCL, already placed functionality in the code, so I'm only re-using
the code. I'm not adding anything new and only using functionality already present in ESXi/KVM/VCL.
All networking is done on hypervisor (ESXi/KVM) and physical switches using VLANs.
>
> I'll try to explain. As I already mentioned before, the code for custom networks is already
present in the VCL. Here is how custom network is added based on vcl.image.project value (already
part of VMware.pm):
>
>         # Add additional Ethernet interfaces if the image project name is not vcl
>         if ($image_project !~ /^vcl$/i && $self->api->can('get_network_names'))
{
>                notify($ERRORS{'DEBUG'}, 0, "image project is: $image_project, checking
if additional network adapters should be configured");
>
>                # Get a list of all the network names configured on the VMware host
>                my @network_names = $self->api->get_network_names();
>                notify($ERRORS{'DEBUG'}, 0, "retrieved network names configured on the
VM host: " . join(", ", @network_names));
>
>                 # Check each network name
>                 # Begin the index at 2 for additional interfaces added because ethernet0
and ethernet1 have already been added
>                for my $network_name (@network_names) {
>                        if ($network_name =~ /$image_project/i || $image_project =~ /$network_name/i)
{
>                                notify($ERRORS{'DEBUG'}, 0, "network name ($network_name)
and image project name ($image_project) intersect, adding network interface to VM for network
$network_name");
>                                %vmx_parameters = (%vmx_parameters, %{$self->get_generated_ethernet_vmx_definition($interface_index,
$network_name)});
>                                $interface_index++;
>                        }
>                        else {
>                                notify($ERRORS{'DEBUG'}, 0, "network name ($network_name)
and image project name ($image_project) do not intersect, network interface will not be added
to VM for network $network_name");
>                        }
>                }
>         }
>         else {
>                notify($ERRORS{'DEBUG'}, 0, "image project is: $image_project, additional
network adapters will not be configured");
>         }
>
> Since vcl.image.project is used by xCAT* modules as well, I didn't want to change it,
so I've added another column vcl.image.vnetworks, which is empty ('') by default or lists
networks separated by comma. Then all listed networks are added during reservation:
>
>         # add custom networks
>         # custom networks listed in vcl.image.networks, separated by ,
>         # network needs to match ESXi Port Group name, which should be already created.
>
>         if ($image_vnetworks) {
>                 notify($ERRORS{'DEBUG'},0, "adding custom networks: $image_vnetworks");
>                 my @vnetwork_names = split(',', $image_vnetworks);
>                 for my $vnetwork_name (@vnetwork_names) {
>                         if ($vnetwork_name) {
>                                 notify($ERRORS{'DEBUG'},0, "adding custom network interface
to VM for network $vnetwork_name");
>                                 %vmx_parameters = (%vmx_parameters, %{$self->get_generated_ethernet_vmx_definition($interface_index,
$vnetwork_name)});
>                                 $interface_index++;
>                         }
>                 }
>         }
>
> Note, that this code doesn't check if network already present on ESXi host, it just adds
it to .vmx file. Hence it's more directed to be used by VCL admin.
>
> VCL administrator needs to add local or global VLANs on network switches used by VCL
and configure Port Groups /w VLAN IDs on VM hosts. Once this is done, VCL admin would add
one or multiple networks to vcl.image.vnetworks per image. Each reservation based on the image
will have additional network interface(s) placed in listed networks. All traffic control is
done by existing network equipment/rules on that VLAN. Reservation will need to follow IP
addressing schema specific to that VLAN (i.e. IP/MASK/GW) and most likely will use existing
DHCP offerings on that VLAN. In case when isolated local VLAN is used there is a need to have
DHCP server configured to provide local IPs. There is no router needed since I need to isolated
traffic to only that VLAN (a).
>
> Let say you make a block allocation for 25 Linux XYZ images for a class. Linux XYZ image
has 'vcl200' listed in vcl.image.vnetworks. When block allocation is on-line, all 25 Linux
XYZ computers will have eth2 interface which is part of vcl200 Port Group on ESXi hosts. (a)
Let's say vcl200 Port Group is configured with VLAN ID 200 which is local to VCL equipment.
This creates isolated network shared by all VMs which are part of vcl200. You can have 192.168.x.0/24
network with no default GW to run DoS attacks within this network. (b) If VLAN ID 200 is global
VLAN and it's used by your desktop PC (or physical servers in your Lab) then your VCL reservation
will be on the same physical network, and will follow the same traffic rules already present
on VLAN 200.
>
> Thanks.
> On Aug 2, 2012, at 11:22 , Georgy Mathew Kallumkal <georgy@ncsu.edu> wrote:
>
>> Hi Dimitri,
>>
>> I couldn't quite understand what you have done. And couldn't find the
>> attachment too.
>>
>> -Georgy
>>
>> On Thu, Aug 2, 2012 at 10:54 AM, Mani Shafa'atDoost <mani.doost@gmail.com>wrote:
>>
>>> Hi Dimitri,
>>>
>>> I had some difficulties to understand what are you going to do. But, as far
>>> as I understood, you are planing to add more than 2 NICs on each VM(
>>> depends on preconfigured network information) and then assign these NICs to
>>> special VLAN group.
>>> I was thinking to use the same method but there are some problem of using
>>> of just VLans:
>>> 1. You can't make a communication between two VMs which are located over
>>> the router by using Vlan.
>>> 2. You can't control traffic and make special rules for traffic of each VM.
>>> 3. This solution isn't extendable if you want to extend VCL in near future.
>>> So I came with the following solution :
>>> http://users.cis.fiu.edu/~mshaf012/pdf/OpenVswitch.pdf
>>>
>>> BTW, I can't see your attachment, it would be better to upload it
>>> somewhere.
>>>
>>> Best Regards
>>> Mani
>>>
>>> On Thu, Aug 2, 2012 at 10:29 AM, Dmitri Chebotarov <dchebota@gmu.edu>
>>> wrote:
>>>
>>>> Hi
>>>>
>>>> I would like to share a method of adding custom networks per a VCL image.
>>>> This solution is based on the code already present in VCL.
>>>>
>>>> This solution is not end-user/student oriented. There is no option to
>>>> specify networks during reservation.
>>>> Network(s) and VM hosts needs to be preconfigured before custom networks
>>>> can be used.
>>>> Switch Local VLAN IDs can be used to create isolated networks for VCL.
>>>> Global VLAN IDs can be used to give VCL reservations access to different
>>>> networks.
>>>>
>>>> Isolated network use example would be a security class where malicious
>>>> traffic needs to be generated and analyzed. Running such class on
>>>> public/private VCL network is not desirable, so an isolated VCL network
>>> can
>>>> be used in this case. In case with isolated networks a DHCP server needs
>>> to
>>>> be present to provide IP addresses on isolated networks. The DHCP server
>>>> can be part of VCL infrastructure or be a very-long-term server
>>> reservation
>>>> (the new feature in VCL 2.3).
>>>>
>>>> Global use example would be a VCL reservation which needs access to a
>>>> department network or lab environment. It allows to connect VCL
>>>> reservations to existing infrastructure.
>>>>
>>>> It's different from CS/OS implementation as it doesn't give any control
>>>> over networking to end-user, hence no security concerns about VCL
>>> end-users
>>>> getting unwanted access to a custom network(s).
>>>> Since custom network(s) is part of the image properties, only
>>> groups/users
>>>> who allowed to make reservation based on the image will have access to
>>>> specified networks.
>>>>
>>>> Below is PDF file with code changes. I've tested it in sandbox env and it
>>>> seems to be working well.
>>>>
>>>>
>>>>
>>>>
>>>> Thanks.
>>>>
>>>> On Jul 31, 2012, at 13:16 , Mani Shafa'atDoost <mani.doost@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi  Dmitri,
>>>>>
>>>>> This seems interesting to me.  I am planing to do a contribution on VCL
>>>>> project which allow you to make a network topology of VMs and load this
>>>>> topology instead of one VM. For this reason I need to add more NIC on
>>>> some
>>>>> images and also I need to make a mechanism for this communication.
>>>>> I have read a lot about this and I think the best option is using Open
>>>>> Vswitch on VCL which has a strong support on networking part and allow
>>>> you
>>>>> to do many things on network side. Currently I am making some documents
>>>> and
>>>>> I will post it here. I would be happy to see some other people from
>>> this
>>>>> group to work on networking part.
>>>>>
>>>>>
>>>>> On Tue, Jul 31, 2012 at 12:55 PM, Dmitri Chebotarov <dchebota@gmu.edu
>>>>> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> Looks like VMware.pm already has a code to add custom networking
based
>>>> on
>>>>>> project name:
>>>>>>
>>>>>>       # Add additional Ethernet interfaces if the image project name
>>> is
>>>>>> not vcl
>>>>>>       if ($image_project !~ /^vcl$/i &&
>>>>>> $self->api->can('get_network_names')) {
>>>>>>               notify($ERRORS{'DEBUG'}, 0, "image project is:
>>>>>> $image_project, checking if additional network adapters should be
>>>>>> configured");
>>>>>>
>>>>>>               # Get a list of all the network names configured on
the
>>>>>> VMware host
>>>>>>               my @network_names = $self->api->get_network_names();
>>>>>>               notify($ERRORS{'DEBUG'}, 0, "retrieved network names
>>>>>> configured on the VM host: " . join(", ", @network_names));
>>>>>>
>>>>>>               # Check each network name
>>>>>>               # Begin the index at 2 for additional interfaces added
>>>>>> because ethernet0 and ethernet1 have already been added
>>>>>>               for my $network_name (@network_names) {
>>>>>>                       if ($network_name =~ /$image_project/i ||
>>>>>> $image_project =~ /$network_name/i) {
>>>>>>                               notify($ERRORS{'DEBUG'}, 0, "network
>>> name
>>>>>> ($network_name) and image project name ($image_project) intersect,
>>>> adding
>>>>>> network interface to VM for network $network_name");
>>>>>>                               %vmx_parameters = (%vmx_parameters,
>>>>>> %{$self->get_generated_ethernet_vmx_definition($interface_index,
>>>>>> $network_name)});
>>>>>>                               $interface_index++;
>>>>>>                       }
>>>>>>                       else {
>>>>>>                               notify($ERRORS{'DEBUG'}, 0, "network
>>> name
>>>>>> ($network_name) and image project name ($image_project) do not
>>>> intersect,
>>>>>> network interface will not be added to VM for network $network_name");
>>>>>>                       }
>>>>>>               }
>>>>>>
>>>>>>       }
>>>>>>       else {
>>>>>>               notify($ERRORS{'DEBUG'}, 0, "image project is:
>>>>>> $image_project, additional network adapters will not be configured");
>>>>>>       }
>>>>>>
>>>>>> By default all image has project as 'vcl'.
>>>>>> Currently project name can only be in ('vcl','hpc','vclhpc').
>>>>>> Will it be OK to remove ENUM on project column, create corresponding
>>>>>> additional networks on ESXi servers and use this option to add custom
>>>>>> networks?
>>>>>>
>>>>>> I've also tested how additional VM networks in Virtual Host profile
>>>> work.
>>>>>> Once I configured profile with additional networks, these networks
are
>>>>>> added to each reservation which starts on corresponding VM host.
>>>>>> This may not be what I'm looking for, as I don't need all these
>>> networks
>>>>>> on each image.
>>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>> On Jul 31, 2012, at 11:39 , Dmitri Chebotarov <dchebota@gmu.edu>
>>> wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> VCL 2.3 has added two more VM Networks in Virtual Host profile.
>>>>>>> Can I use these networks (all at once or selectively) in images?
>>>>>>>
>>>>>>> I'm looking to add custom network interface to a Linux image
(in
>>>>>> addition to default Private/Public).
>>>>>>>
>>>>>>> --
>>>>>>> Thank you,
>>>>>>>
>>>>>>> Dmitri Chebotarov
>>>>>>> Virtual Computing Lab Systems Engineer, TSD - Ent Servers &
Messaging
>>>>>>> 223 Aquia Building, Ffx, MSN: 1B5
>>>>>>> Phone: (703) 993-6175
>>>>>>> Fax: (703) 993-3404
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Thank you,
>>>>>>
>>>>>> Dmitri Chebotarov
>>>>>> Virtual Computing Lab Systems Engineer, TSD - Ent Servers & Messaging
>>>>>> 223 Aquia Building, Ffx, MSN: 1B5
>>>>>> Phone: (703) 993-6175
>>>>>> Fax: (703) 993-3404
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards
>>>>> Mani
>>>>
>>>>
>>>>
>>>> --
>>>> Thank you,
>>>>
>>>> Dmitri Chebotarov
>>>> Virtual Computing Lab Systems Engineer, TSD - Ent Servers & Messaging
>>>> 223 Aquia Building, Ffx, MSN: 1B5
>>>> Phone: (703) 993-6175
>>>> Fax: (703) 993-3404
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Best Regards
>>> Mani
>>>
>
>
>
> --
> Thank you,
>
> Dmitri Chebotarov
> Virtual Computing Lab Systems Engineer, TSD - Ent Servers & Messaging
> 223 Aquia Building, Ffx, MSN: 1B5
> Phone: (703) 993-6175
> Fax: (703) 993-3404
>
>
>
>

Mime
View raw message