cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcus <shadow...@gmail.com>
Subject Re: VPC's VR missing public NIC eth1
Date Fri, 30 May 2014 16:27:24 GMT
It did. It was changed to accomodate vxlan



On Fri, May 30, 2014 at 10:25 AM, Daan Hoogland <daan.hoogland@gmail.com>
wrote:

> and
> broadcastUriToNicNum.put(
> BroadcastDomainType.Vlan.toUri(Vlan.UNTAGGED).toString(),
> devNum);
>
> ahould read
>
> broadcastUriToNicNum.put(Vlan.UNTAGGED,devNum);
>
>
>
> On Fri, May 30, 2014 at 6:24 PM, Daan Hoogland <daan.hoogland@gmail.com>
> wrote:
> > Or maybe it shouldn't be filtered by BroadcastDomainType.VLAN when it
> > is 'untagged', ...
> >
> > On Fri, May 30, 2014 at 6:23 PM, Daan Hoogland <daan.hoogland@gmail.com>
> wrote:
> >> BroadcastDomainType.Vlan.
> >> toUri(Vlan.UNTAGGED).toString() should return 'untagged' in my opinion
> >>
> >> On Fri, May 30, 2014 at 6:21 PM, Marcus <shadowsor@gmail.com> wrote:
> >>> The issue is that we put this in the map:
> >>>
> >>>
> broadcastUriToNicNum.put(BroadcastDomainType.Vlan.toUri(Vlan.UNTAGGED).toString(),
> >>> devNum);
> >>>
> >>> Which is 'vlan://untagged', and we fetch the eth devNum back by doing:
> >>>
> >>> broadcastUriToNicNum.get(ip.getBroadcastUri());
> >>>
> >>> which is just 'untagged'
> >>>
> >>>
> >>> On Fri, May 30, 2014 at 10:19 AM, Marcus <shadowsor@gmail.com> wrote:
> >>>
> >>>> It's not valid if you've got code that says does string
> 'vlan://untagged'
> >>>> equal 'untagged'.
> >>>>
> >>>>
> >>>> On Fri, May 30, 2014 at 10:17 AM, Daan Hoogland <
> daan.hoogland@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> CLOUDSTACK-5505 looks alright. As for the solution; Isn't 'untagged'
> a
> >>>>> valid uri in itself? I would expect it would have always the value
> >>>>> without the 'vlan://'. That said a solution is better then no
> >>>>> solution, maybe the db upgrade path is best.
> >>>>>
> >>>>> Andrija, can you try as Marcus suggests, editing the db to change
> >>>>> 'untagged' to 'vlan://untagged'?
> >>>>>
> >>>>> thanks,
> >>>>> Daan
> >>>>>
> >>>>> On Fri, May 30, 2014 at 6:03 PM, Marcus <shadowsor@gmail.com> wrote:
> >>>>> > If that works, then I think the fix is better to update the
> database
> >>>>> > upgrade script to look for this and change the vlan_id. That way
> new
> >>>>> > installs and upgrades have consistent data. We could fix it in the
> code
> >>>>> by
> >>>>> > filtering it through
> >>>>> BroadcastDomainType.Vlan.toUri(ipAddr.getVlanTag()),
> >>>>> > but then later when someone expects the vlan id to be in URI format
> >>>>> > somewhere else it will randomly break on people who did an
> upgrade. The
> >>>>> fix
> >>>>> > to CLOUDSTACK-5505 did this, avoiding the mismatch in format in
> the DB
> >>>>> by
> >>>>> > fetching the data from elsewhere that filtered it into URI format,
> and
> >>>>> then
> >>>>> > we just ended up hitting it again here.
> >>>>> >
> >>>>> >
> >>>>> > On Fri, May 30, 2014 at 9:57 AM, Marcus <shadowsor@gmail.com>
> wrote:
> >>>>> >
> >>>>> >> Actually, if you're in the position to play a bit...  New
> deployments
> >>>>> seem
> >>>>> >> to work, and I believe it's because that broadcastUri is stored
> in the
> >>>>> db
> >>>>> >> in the new format:
> >>>>> >>
> >>>>> >> mysql> select id,vlan_id from vlan where network_id = (select id
> from
> >>>>> >> networks where traffic_type="Public");
> >>>>> >> +----+-----------------+
> >>>>> >> | id | vlan_id         |
> >>>>> >> +----+-----------------+
> >>>>> >> |  1 | vlan://untagged |
> >>>>> >> +----+-----------------+
> >>>>> >> 1 row in set (0.00 sec)
> >>>>> >>
> >>>>> >>
> >>>>> >> I believe 4.2 and earlier would say just 'untagged' there. If you
> want
> >>>>> to
> >>>>> >> attempt changing that value to include vlan:// (if it is in fact
> >>>>> missing),
> >>>>> >> then restarting everything, that may fix the  issue.
> >>>>> >>
> >>>>> >>
> >>>>> >>
> >>>>> >> On Fri, May 30, 2014 at 9:38 AM, Marcus <shadowsor@gmail.com>
> wrote:
> >>>>> >>
> >>>>> >>> I thinnk the commit that caused the change was this or related
> to it.
> >>>>> Is
> >>>>> >>> there any way you could test a fix? Do you need me to build 4.3
> RPMs
> >>>>> or can
> >>>>> >>> I just provide a patch? What works for you?
> >>>>> >>>
> >>>>> >>> commit 53d09c6f1843f04c5f1ab76be9419f5584302d1e
> >>>>> >>> Date:   Mon Aug 5 11:52:40 2013 +0200
> >>>>> >>>
> >>>>> >>>     uri code per broadcast/isolation type , default is to accept
> >>>>> anything
> >>>>> >>> as uri , vlan and lswitch need some extra tlc
> >>>>> >>>
> >>>>> >>>
> >>>>> >>>
> >>>>> >>> On Fri, May 30, 2014 at 9:36 AM, Marcus <shadowsor@gmail.com>
> wrote:
> >>>>> >>>
> >>>>> >>>> Note the differences in broadcastUri, here is your plug command:
> >>>>> >>>>
> >>>>> >>>> {
> >>>>> >>>>     "com.cloud.agent.api.PlugNicCommand": {
> >>>>> >>>>         "nic": {
> >>>>> >>>>             "deviceId": 1,
> >>>>> >>>>             "networkRateMbps": 99999,
> >>>>> >>>>             "defaultNic": true,
> >>>>> >>>>             "uuid": "6c782af3-2071-4543-acdc-cb30096e89ff",
> >>>>> >>>>             "ip": "46.232.xxx.246",
> >>>>> >>>>             "netmask": "255.255.255.0",
> >>>>> >>>>             "gateway": "46.232.xxx.1",
> >>>>> >>>>             "mac": "06:53:82:00:00:25",
> >>>>> >>>>             "broadcastType": "Vlan",
> >>>>> >>>>             "type": "Public",
> >>>>> >>>>             "broadcastUri": "vlan://untagged",
> >>>>> >>>>             "isolationUri": "vlan://untagged",
> >>>>> >>>>             "isSecurityGroupEnabled": false,
> >>>>> >>>>             "name": "breth1-500"
> >>>>> >>>>         },
> >>>>> >>>>         "instanceName": "r-801-VM",
> >>>>> >>>>         "vmType": "DomainRouter",
> >>>>> >>>>         "wait": 0
> >>>>> >>>>     }
> >>>>> >>>> }
> >>>>> >>>>
> >>>>> >>>> and here is your ip associate command:
> >>>>> >>>>
> >>>>> >>>> {
> >>>>> >>>>     "com.cloud.agent.api.routing.IpAssocVpcCommand": {
> >>>>> >>>>         "ipAddresses": [
> >>>>> >>>>             {
> >>>>> >>>>                 "accountId": 11,
> >>>>> >>>>                 "publicIp": "46.232.xxx.246",
> >>>>> >>>>                 "sourceNat": true,
> >>>>> >>>>                 "add": true,
> >>>>> >>>>                 "oneToOneNat": false,
> >>>>> >>>>                 "firstIP": false,
> >>>>> >>>>                 "broadcastUri": "untagged",
> >>>>> >>>>                 "vlanGateway": "46.232.xxx.1",
> >>>>> >>>>                 "vlanNetmask": "255.255.255.0",
> >>>>> >>>>                 "vifMacAddress": "06:53:82:00:00:25",
> >>>>> >>>>                 "networkRate": 99999,
> >>>>> >>>>                  "trafficType": "Public",
> >>>>> >>>>                 "networkName": "breth1-500"
> >>>>> >>>>             }
> >>>>> >>>>         ],
> >>>>> >>>>         "accessDetails": {
> >>>>> >>>>             "router.guest.ip": "46.232.xxx.246",
> >>>>> >>>>             "zone.network.type": "Advanced",
> >>>>> >>>>             "router.name": "r-801-VM",
> >>>>> >>>>             "router.ip": "169.254.0.52"
> >>>>> >>>>         },
> >>>>> >>>>         "wait": 0
> >>>>> >>>>     }
> >>>>> >>>> }
> >>>>> >>>>
> >>>>> >>>>
> >>>>> >>>> On Fri, May 30, 2014 at 9:27 AM, Andrija Panic <
> >>>>> andrija.panic@gmail.com>
> >>>>> >>>> wrote:
> >>>>> >>>>
> >>>>> >>>>> yes, correct, eth1 is present, and can be started by static IP
> >>>>> >>>>> configuration...
> >>>>> >>>>>
> >>>>> >>>>>
> >>>>> >>>>> On 30 May 2014 17:25, Marcus <shadowsor@gmail.com> wrote:
> >>>>> >>>>>
> >>>>> >>>>> > Let me make sure I understand... the 'Plug' of the nic works
> >>>>> fine, as
> >>>>> >>>>> it
> >>>>> >>>>> > seems you do have an eth1 and can manually assign the IP to
> get
> >>>>> it to
> >>>>> >>>>> work?
> >>>>> >>>>> > If that's the case then there's probably not an issue in
> >>>>> >>>>> BridgeVifDriver or
> >>>>> >>>>> > the XML. It is definitely in fetching/matching the eth
> device here
> >>>>> >>>>> >
> >>>>> >>>>> > vpc_ipassoc.sh 169.254.0.52  -A  -l 46.232.xxx.246 -c
> ethnull -g
> >>>>> >>>>> > 46.232.xxx.1 -m 24 -n 46.232.xxx.0
> >>>>> >>>>> >
> >>>>> >>>>> >
> >>>>> >>>>> > On Fri, May 30, 2014 at 8:33 AM, Andrija Panic <
> >>>>> >>>>> andrija.panic@gmail.com>
> >>>>> >>>>> > wrote:
> >>>>> >>>>> >
> >>>>> >>>>> > > Nope, started, did check, it is reported now as Debian 5
> VM, but
> >>>>> >>>>> still
> >>>>> >>>>> > > doesn't work...
> >>>>> >>>>> > > Rebooted VPC (destroyed VR, new one created...)
> >>>>> >>>>> > >
> >>>>> >>>>> > > $ virsh dumpxml r-812-VM
> >>>>> >>>>> > >  ...
> >>>>> >>>>> > > <description>Debian GNU/Linux 5(64-bit)</description>
> >>>>> >>>>> > > ...
> >>>>> >>>>> > > <os>
> >>>>> >>>>> > >     <type arch='x86_64' machine='rhel6.5.0'>hvm</type>
> >>>>> >>>>> > >     <boot dev='cdrom'/>
> >>>>> >>>>> > >     <boot dev='hd'/>
> >>>>> >>>>> > >   </os>
> >>>>> >>>>> > >
> >>>>> >>>>> > > :(
> >>>>> >>>>> > >
> >>>>> >>>>> > >
> >>>>> >>>>> > >
> >>>>> >>>>> > > On 30 May 2014 16:24, Andrija Panic <
> andrija.panic@gmail.com>
> >>>>> >>>>> wrote:
> >>>>> >>>>> > >
> >>>>> >>>>> > > > I confirm, the highest is Debian 5 64bit
> >>>>> >>>>> > > >
> >>>>> >>>>> > > > Per docs
> >>>>> >>>>> > > >
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.3/rnotes.html#upgrade-from-4-2-x-to-4-3
> >>>>> >>>>> > > > , you should use Debian 7.0 64bit as OS type for
> >>>>> system-kvm-4.3
> >>>>> >>>>> > > template...
> >>>>> >>>>> > > > (same for xen and vmware templatest)
> >>>>> >>>>> > > >
> >>>>> >>>>> > > > Will change now DB to point to debian 5, and let you
> know.
> >>>>> >>>>> > > >
> >>>>> >>>>> > > > This is guest_os_id:          184 |           2 | NULL |
> >>>>> >>>>> > > > 986a0e98-39d6-11e3-8f93-0025904e4412 | Debian GNU/Linux
> >>>>> 7(64-bit)
> >>>>> >>>>> > > >
> >>>>> >>>>> > > >
> >>>>> >>>>> > > > On 30 May 2014 15:45, Joris van Lieshout <
> >>>>> >>>>> > > JvanLieshout@schubergphilis.com>
> >>>>> >>>>> > > > wrote:
> >>>>> >>>>> > > >
> >>>>> >>>>> > > >> Andrija,
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >> The thing is I don't know who the os matching on KVM
> works.
> >>>>> >>>>> There must
> >>>>> >>>>> > > be
> >>>>> >>>>> > > >> a way to list supported os types.
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >> I also did some queuing on the guest_os_hypervisor
> table (ACS
> >>>>> >>>>> 4.3)
> >>>>> >>>>> > and I
> >>>>> >>>>> > > >> don't see Debian 7 for KVM listed.
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >> select * from guest_os join guest_os_hypervisor on
> >>>>> >>>>> > > >> guest_os.id=guest_os_hypervisor.guest_os_id where
> >>>>> >>>>> > > >> guest_os_hypervisor.hypervisor_type='KVM' and
> >>>>> >>>>> > > >> guest_os_hypervisor.guest_os_name like '%Debian%';
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >> What was the guest_os_id you where using? Could you try
> id 72
> >>>>> >>>>> (Debian
> >>>>> >>>>> > 5
> >>>>> >>>>> > > >> 64-bit)? Adjust both os_type_id in vm_instance and
> >>>>> vm_template
> >>>>> >>>>> (where
> >>>>> >>>>> > > >> type='SYSTEM' and hypervisor_type='KVM').
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >> Kind regards,
> >>>>> >>>>> > > >> Joris van Lieshout
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >> Schuberg Philis
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >> On 30/05/14 15:33, "Andrija Panic" <
> andrija.panic@gmail.com>
> >>>>> >>>>> wrote:
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >> >Joris,
> >>>>> >>>>> > > >> >
> >>>>> >>>>> > > >> >do you have recommendation on how in particular to try
> ? I'm
> >>>>> >>>>> not sure
> >>>>> >>>>> > > how
> >>>>> >>>>> > > >> >to fix that, except playing with editing systemvm-4.3
> >>>>> template
> >>>>> >>>>> to
> >>>>> >>>>> > > define
> >>>>> >>>>> > > >> >it
> >>>>> >>>>> > > >> >as another OS type... ?
> >>>>> >>>>> > > >> >
> >>>>> >>>>> > > >> >Thanks again,
> >>>>> >>>>> > > >> >Andrija
> >>>>> >>>>> > > >> >
> >>>>> >>>>> > > >> >
> >>>>> >>>>> > > >> >On 30 May 2014 15:30, Joris van Lieshout <
> >>>>> >>>>> > > >> JvanLieshout@schubergphilis.com>
> >>>>> >>>>> > > >> >wrote:
> >>>>> >>>>> > > >> >
> >>>>> >>>>> > > >> >> I've read back a bit in the code and if you look at
> >>>>> >>>>> > > >> BridgeVifDriver.java
> >>>>> >>>>> > > >> >> (this is where the log message with the nic profile
> is
> >>>>> >>>>> generated)
> >>>>> >>>>> > you
> >>>>> >>>>> > > >> >>can
> >>>>> >>>>> > > >> >> see that the nic information might be off already
> once ACS
> >>>>> >>>>> hits the
> >>>>> >>>>> > > >> >> LibvirtVMDef.InterfaceDef plug function. This leads
> be to
> >>>>> >>>>> believer
> >>>>> >>>>> > > that
> >>>>> >>>>> > > >> >> the HVM/PV OS mismatch issue might still be related.
> Try
> >>>>> >>>>> fixing
> >>>>> >>>>> > that
> >>>>> >>>>> > > >> >> first. At least it will allow us to exclude this
> from the
> >>>>> >>>>> list.
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >> Kind regards,
> >>>>> >>>>> > > >> >> Joris van Lieshout
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >> Schuberg Philis
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >> On 30/05/14 15:26, "Andrija Panic" <
> >>>>> andrija.panic@gmail.com>
> >>>>> >>>>> > wrote:
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >> >OK, thanks Joris.
> >>>>> >>>>> > > >> >> >
> >>>>> >>>>> > > >> >> >I will try playing with OS version option, on the
> >>>>> >>>>> systemvm-kvm-4.3
> >>>>> >>>>> > > >> >> >template...
> >>>>> >>>>> > > >> >> >
> >>>>> >>>>> > > >> >> >Let me know if I can help with anything more.
> >>>>> >>>>> > > >> >> >
> >>>>> >>>>> > > >> >> >Thanks.
> >>>>> >>>>> > > >> >> >Andrija
> >>>>> >>>>> > > >> >> >
> >>>>> >>>>> > > >> >> >
> >>>>> >>>>> > > >> >> >On 30 May 2014 15:19, Joris van Lieshout
> >>>>> >>>>> > > >> >><JvanLieshout@schubergphilis.com
> >>>>> >>>>> > > >> >> >
> >>>>> >>>>> > > >> >> >wrote:
> >>>>> >>>>> > > >> >> >
> >>>>> >>>>> > > >> >> >> Hi Andrija,
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >> That does sound familiar and in the start xml of
> KVM
> >>>>> you
> >>>>> >>>>> can see
> >>>>> >>>>> > > >> >>"<type
> >>>>> >>>>> > > >> >> >> arch='x86_64' machine='pc'>hvm</type>". I don't
> know
> >>>>> >>>>> KVM+ACS
> >>>>> >>>>> > well
> >>>>> >>>>> > > >> >>enough
> >>>>> >>>>> > > >> >> >> to judge if this is the cause but I thing
> focusing on
> >>>>> >>>>> getting
> >>>>> >>>>> > the
> >>>>> >>>>> > > VR
> >>>>> >>>>> > > >> >> >> started as PV guest might be worth trying. On the
> other
> >>>>> >>>>> hand I
> >>>>> >>>>> > do
> >>>>> >>>>> > > >> see
> >>>>> >>>>> > > >> >> >> patchviasocket.pl being executed successfully...
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >> The other thing I see is, and now we're getting
> into
> >>>>> java
> >>>>> >>>>> code,
> >>>>> >>>>> > is
> >>>>> >>>>> > > >> >>this:
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >> 2014-05-30 14:41:01,386{GMT} DEBUG
> >>>>> >>>>> > [kvm.resource.BridgeVifDriver]
> >>>>> >>>>> > > >> >> >> (agentRequest-Handler-3:)
> >>>>> >>>>> > > >> >> >>nic=[Nic:Public-46.232.xxx.246-vlan://untagged]
> >>>>> >>>>> > > >> >> >> 2014-05-30 14:41:01,502{GMT} DEBUG
> [cloud.agent.Agent]
> >>>>> >>>>> > > >> >> >> (agentRequest-Handler-3:) Processing command:
> >>>>> >>>>> > > >> >> >> com.cloud.agent.api.routing.IpAssocVpcCommand
> >>>>> >>>>> > > >> >> >> 2014-05-30 14:41:01,506{GMT} DEBUG
> >>>>> >>>>> > > >> >> >> [resource.virtualnetwork.VirtualRoutingResource]
> >>>>> >>>>> > > >> >> >>(agentRequest-Handler-3:)
> >>>>> >>>>> > > >> >> >> Executing:
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> >
> /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh
> >>>>> >>>>> > > >> >> >> vpc_ipassoc.sh 169.254.0.52  -A  -l
> 46.232.xxx.246 -c
> >>>>> >>>>> ethnull -g
> >>>>> >>>>> > > >> >> >> 46.232.xxx.1 -m 24 -n 46.232.xxx.0
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >> My suspicion is that somewhere in the translation
> from
> >>>>> the
> >>>>> >>>>> nic
> >>>>> >>>>> > > >> >>profile
> >>>>> >>>>> > > >> >> >>to
> >>>>> >>>>> > > >> >> >> the actual route_proxy.sh command ACS failes to
> find
> >>>>> the
> >>>>> >>>>> nic id
> >>>>> >>>>> > > and
> >>>>> >>>>> > > >> >> >> returns null.
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >> Let me dig a bit deeper and see what I can find
> but
> >>>>> this is
> >>>>> >>>>> > where
> >>>>> >>>>> > > we
> >>>>> >>>>> > > >> >> >>might
> >>>>> >>>>> > > >> >> >> need some help from someone with knowledge of this
> >>>>> pice of
> >>>>> >>>>> the
> >>>>> >>>>> > > code.
> >>>>> >>>>> > > >> >>:)
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >> Kind regards,
> >>>>> >>>>> > > >> >> >> Joris van Lieshout
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >> Schuberg Philis
> >>>>> >>>>> > > >> >> >> Boeingavenue 271
> >>>>> >>>>> > > >> >> >> 1119 PD  Schiphol-Rijk
> >>>>> >>>>> > > >> >> >> schubergphilis.com
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >> +31 20-7506672
> >>>>> >>>>> > > >> >> >> +31 6-51428188
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >> On 30/05/14 14:49, "Andrija Panic" <
> >>>>> >>>>> andrija.panic@gmail.com>
> >>>>> >>>>> > > wrote:
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >> >Hi Joris,
> >>>>> >>>>> > > >> >> >> >
> >>>>> >>>>> > > >> >> >> >I have turned on DEBUG loging in agent.log on
> >>>>> cs1.xxx/net
> >>>>> >>>>> host:
> >>>>> >>>>> > > >> >> >> >
> >>>>> >>>>> > > >> >> >> >So, management logs again:
> >>>>> http://pastebin.com/F6BRf7Y9
> >>>>> >>>>> > > >> >> >> >Agent logs on cs1.xxx:
> http://pastebin.com/BJauKbaC
> >>>>> >>>>> > > >> >> >> >
> >>>>> >>>>> > > >> >> >> >Not playing smart, but there is some error:
> >>>>> >>>>> > > >> >> >> >[kvm.resource.KVMGuestOsMapper]
> >>>>> >>>>> > > >> >> >> >(agentRequest-Handler-3:) Can't find the mapping
> of
> >>>>> guest
> >>>>> >>>>> os:
> >>>>> >>>>> > > >> Debian
> >>>>> >>>>> > > >> >> >> >GNU/Linux 7(64-bit)
> >>>>> >>>>> > > >> >> >> >
> >>>>> >>>>> > > >> >> >> >Best,
> >>>>> >>>>> > > >> >> >> >Andrija
> >>>>> >>>>> > > >> >> >> >
> >>>>> >>>>> > > >> >> >> >
> >>>>> >>>>> > > >> >> >> >On 30 May 2014 14:26, Joris van Lieshout
> >>>>> >>>>> > > >> >> >><JvanLieshout@schubergphilis.com
> >>>>> >>>>> > > >> >> >> >
> >>>>> >>>>> > > >> >> >> >wrote:
> >>>>> >>>>> > > >> >> >> >
> >>>>> >>>>> > > >> >> >> >> Hi Andrija,
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> Bold formatting does not come trough on the dev
> >>>>> list. :)
> >>>>> >>>>> > > >> >> >> >> But u might need a bit more info.
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> At a certain point I see this line
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> 2014-05-30 13:56:23,935 DEBUG [c.c.a.t.Request]
> >>>>> >>>>> > > >> >> >> >> (Job-Executor-77:ctx-ec3d358e ctx-f35b12af) Seq
> >>>>> >>>>> 1-609104082:
> >>>>> >>>>> > > >> >>Sending
> >>>>> >>>>> > > >> >> >> {
> >>>>> >>>>> > > >> >> >> >> Cmd , MgmtId: 161344838950, via: 1(
> cs1.xxxxx.net),
> >>>>> >>>>> Ver: v1,
> >>>>> >>>>> > > >> Flags:
> >>>>> >>>>> > > >> >> >> >>100111,
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>[{"com.cloud.agent.api.StartCommand":{"vm":{"id":801,"name":"r-801-VM
> >>>>> >>>>> > > >> >>>>>>".
> >>>>> >>>>> > > >> >> >>>>..
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> This is where the information is passed on to
> the
> >>>>> agent
> >>>>> >>>>> > > handles.
> >>>>> >>>>> > > >> >>For
> >>>>> >>>>> > > >> >> >>XS
> >>>>> >>>>> > > >> >> >> >> this would initiate an agent handler on the
> >>>>> management
> >>>>> >>>>> server
> >>>>> >>>>> > > but
> >>>>> >>>>> > > >> >>for
> >>>>> >>>>> > > >> >> >> >>KVM,
> >>>>> >>>>> > > >> >> >> >> if I remember correctly, it passed the command
> on
> >>>>> to the
> >>>>> >>>>> > > >> >>cloudstack
> >>>>> >>>>> > > >> >> >> >>agent
> >>>>> >>>>> > > >> >> >> >> service on the hypervisor.
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> Can you check the cloud service log on the KVM
> >>>>> >>>>> hypervisor
> >>>>> >>>>> > > >> >>executing
> >>>>> >>>>> > > >> >> >>the
> >>>>> >>>>> > > >> >> >> >> request? it's this server cs1.xxxxx.net and
> then
> >>>>> >>>>> search top
> >>>>> >>>>> > > down
> >>>>> >>>>> > > >> >>for
> >>>>> >>>>> > > >> >> >> >> 609104082 in the log. See if you can provide
> the log
> >>>>> >>>>> from the
> >>>>> >>>>> > > >> >>agent
> >>>>> >>>>> > > >> >> >> >> handler thread started by that sequence.
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> Kind regards,
> >>>>> >>>>> > > >> >> >> >> Joris van Lieshout
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> Schuberg Philis
> >>>>> >>>>> > > >> >> >> >> Boeingavenue 271
> >>>>> >>>>> > > >> >> >> >> 1119 PD  Schiphol-Rijk
> >>>>> >>>>> > > >> >> >> >> schubergphilis.com
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> +31 20-7506672
> >>>>> >>>>> > > >> >> >> >> +31 6-51428188
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> On 30/05/14 14:08, "Andrija Panic" <
> >>>>> >>>>> andrija.panic@gmail.com>
> >>>>> >>>>> > > >> >>wrote:
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >Hi Joris,
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >here is the management log:
> >>>>> >>>>> http://pastebin.com/zxnKxFhk
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >Interesting parts (to me): in bold
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >2014-05-30 13:56:21,899 DEBUG
> >>>>> >>>>> > > >> >>[o.a.c.s.m.AncientDataMotionStrategy]
> >>>>> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af)
> >>>>> copyAsync
> >>>>> >>>>> > > inspecting
> >>>>> >>>>> > > >> >>src
> >>>>> >>>>> > > >> >> >> >>type
> >>>>> >>>>> > > >> >> >> >> >TEMPLATE copyAsync inspecting dest type VOLUME
> >>>>> >>>>> > > >> >> >> >> >2014-05-30 13:56:21,905 DEBUG
> [c.c.a.t.Request]
> >>>>> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af)
> Seq
> >>>>> >>>>> > 4-1248669612:
> >>>>> >>>>> > > >> >> >>Sending
> >>>>> >>>>> > > >> >> >> >>{
> >>>>> >>>>> > > >> >> >> >> >Cmd , MgmtId: 161344838950, via: 4(
> cs2.xxxxx.net),
> >>>>> >>>>> Ver: v1,
> >>>>> >>>>> > > >> >>Flags:
> >>>>> >>>>> > > >> >> >> >> 100011,
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org
> >>>>> >>>>> > > >> >>>>>>>.a
> >>>>> >>>>> > > >> >> >>>>>pa
> >>>>> >>>>> > > >> >> >> >>>ch
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>e.cloudstack.storage.to.TemplateObjectTO":{"path":"1adc1d2e-56ae-4a0
> >>>>> >>>>> > > >> >>>>>>>f-
> >>>>> >>>>> > > >> >> >>>>>b0
> >>>>> >>>>> > > >> >> >> >>>b4
> >>>>> >>>>> > > >> >> >> >> >-5e351e7cae55","origUrl":"
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> http://download.cloud.com/templates/4.3/systemvm64template-2014-01-14-mas
> >>>>> >>>>> > > >> >> >> >>t
> >>>>> >>>>> > > >> >> >> >> >er-kvm.qcow2.bz2
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>","uuid":"1adc1d2e-56ae-4a0f-b0b4-5e351e7cae55","id":414,"format":"Q
> >>>>> >>>>> > > >> >>>>>>>CO
> >>>>> >>>>> > > >> >> >>>>>W2
> >>>>> >>>>> > > >> >> >> >>>",
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> >
> >"accountId":2,"checksum":"85a1bed07bf43cbf022451cb2ecae4ff","
> >>>>> >>>>> > > >> >> >> >> >*hvm":true*
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>,"displayText":"systemvm-kvm-4.3","imageDataStore":{"org.apache.clou
> >>>>> >>>>> > > >> >>>>>>>ds
> >>>>> >>>>> > > >> >> >>>>>ta
> >>>>> >>>>> > > >> >> >> >>>ck
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>.storage.to.PrimaryDataStoreTO":{"uuid":"5b93422e-1a66-353d-88a8-220
> >>>>> >>>>> > > >> >>>>>>>3f
> >>>>> >>>>> > > >> >> >>>>>79
> >>>>> >>>>> > > >> >> >> >>>b1
> >>>>> >>>>> > > >> >> >> >> >dc6","id":209,"poolType":"RBD","host":"
> >>>>> >>>>> > > >> >> >> >> >cephmon.xxxxx.net
> >>>>> >>>>> > > >> ","path":"cloudstack","port":6789,"url":"RBD://
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> cephmon.xxxxx.net/cloudstack/?ROLE=Primary&STOREUUID=5b93422e-1a66-353d-8
> >>>>> >>>>> > > >> >> >> >>8
> >>>>> >>>>> > > >> >> >> >> >a8-2203f79b1dc6
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>"}},"name":"414-2-ec331e74-5858-3153-91a9-1d706d9c533e","hypervisorT
> >>>>> >>>>> > > >> >>>>>>>yp
> >>>>> >>>>> > > >> >> >>>>>e"
> >>>>> >>>>> > > >> >> >> >>>:"
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>KVM"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{
> >>>>> >>>>> > > >> >>>>>>>"u
> >>>>> >>>>> > > >> >> >>>>>ui
> >>>>> >>>>> > > >> >> >> >>>d"
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>:"9c440d3b-cba5-4960-b8bf-dca90291cd2b","volumeType":"ROOT","dataSto
> >>>>> >>>>> > > >> >>>>>>>re
> >>>>> >>>>> > > >> >> >>>>>":
> >>>>> >>>>> > > >> >> >> >>>{"
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"5b9342
> >>>>> >>>>> > > >> >>>>>>>2e
> >>>>> >>>>> > > >> >> >>>>>-1
> >>>>> >>>>> > > >> >> >> >>>a6
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> >6-353d-88a8-2203f79b1dc6","id":209,"poolType":"RBD","host":"
> >>>>> >>>>> > > >> >> >> >> >cephmon.xxxxx.net
> >>>>> >>>>> > > >> ","path":"cloudstack","port":6789,"url":"RBD://
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> cephmon.xxxxx.net/cloudstack/?ROLE=Primary&STOREUUID=5b93422e-1a66-353d-8
> >>>>> >>>>> > > >> >> >> >>8
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>a8-2203f79b1dc6"}},"name":"ROOT-801","size":2621440000,"volumeId":10
> >>>>> >>>>> > > >> >>>>>>>64
> >>>>> >>>>> > > >> >> >>>>>,"
> >>>>> >>>>> > > >> >> >> >>>vm
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>Name":"r-801-VM","accountId":11,"format":"RAW","id":1064,"deviceId":
> >>>>> >>>>> > > >> >>>>>>>0,
> >>>>> >>>>> > > >> >> >>>>>"h
> >>>>> >>>>> > > >> >> >> >>>yp
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>ervisorType":"KVM"}},"executeInSequence":false,"options":{},"wait":0}}
> >>>>> >>>>> > > >> >>>>>]
> >>>>> >>>>> > > >> >> >> >> >}
> >>>>> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,742 DEBUG
> [c.c.a.t.Request]
> >>>>> >>>>> > > >> >> >> >> >(AgentManager-Handler-12:null) Seq
> 4-1248669612:
> >>>>> >>>>> Processing:
> >>>>> >>>>> > >  {
> >>>>> >>>>> > > >> >> >>Ans: ,
> >>>>> >>>>> > > >> >> >> >> >MgmtId: 161344838950, via: 4, Ver: v1, Flags:
> 10,
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{
> >>>>> >>>>> > > >> >>>>>>>"o
> >>>>> >>>>> > > >> >> >>>>>rg
> >>>>> >>>>> > > >> >> >> >>>.a
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>pache.cloudstack.storage.to.VolumeObjectTO":{"size":2621440000,"path
> >>>>> >>>>> > > >> >>>>>>>":
> >>>>> >>>>> > > >> >> >>>>>"9
> >>>>> >>>>> > > >> >> >> >>>c4
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>40d3b-cba5-4960-b8bf-dca90291cd2b","accountId":0,"format":"RAW","id"
> >>>>> >>>>> > > >> >>>>>>>:0
> >>>>> >>>>> > > >> >> >>>>>}}
> >>>>> >>>>> > > >> >> >> >>>,"
> >>>>> >>>>> > > >> >> >> >> >result":true,"wait":0}}]
> >>>>> >>>>> > > >> >> >> >> >}
> >>>>> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,742 DEBUG
> [c.c.a.t.Request]
> >>>>> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af)
> Seq
> >>>>> >>>>> > 4-1248669612:
> >>>>> >>>>> > > >> >> >> >>Received:  {
> >>>>> >>>>> > > >> >> >> >> >Ans: , MgmtId: 161344838950, via: 4, Ver: v1,
> >>>>> Flags:
> >>>>> >>>>> 10, {
> >>>>> >>>>> > > >> >> >> >>CopyCmdAnswer
> >>>>> >>>>> > > >> >> >> >> >} }
> >>>>> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,773 DEBUG
> >>>>> >>>>> > > >> >> >> >>
> >[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
> >>>>> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af)
> >>>>> Removing
> >>>>> >>>>> nic
> >>>>> >>>>> > > >> >>NicProfile[
> >>>>> >>>>> > > >> >> >> >>
> >*1092-801-null*-46.232.xxx.246-vlan://untagged of
> >>>>> type
> >>>>> >>>>> > Public
> >>>>> >>>>> > > >> >>from
> >>>>> >>>>> > > >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >nics
> >>>>> >>>>> > > >> >> >> >> >passed on vm start. The nic will be plugged
> later
> >>>>> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,773 DEBUG
> >>>>> >>>>> > > >> >> >> >>
> >[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
> >>>>> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af)
> >>>>> Removing
> >>>>> >>>>> nic
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>NicProfile[1093-801-cd9fd29a-0573-4715-8742-00ecb9f82c9d-10.0.1.1-vl
> >>>>> >>>>> > > >> >>>>>>>an
> >>>>> >>>>> > > >> >> >>>>>:/
> >>>>> >>>>> > > >> >> >> >>>/4
> >>>>> >>>>> > > >> >> >> >> >4
> >>>>> >>>>> > > >> >> >> >> >of type Guest from the nics passed on vm
> start. The
> >>>>> >>>>> nic will
> >>>>> >>>>> > > be
> >>>>> >>>>> > > >> >> >>plugged
> >>>>> >>>>> > > >> >> >> >> >later
> >>>>> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,773 DEBUG
> >>>>> >>>>> > > >> >> >> >>
> >[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
> >>>>> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af)
> >>>>> Removing
> >>>>> >>>>> nic
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>NicProfile[1094-801-cd9fd29a-0573-4715-8742-00ecb9f82c9d-10.0.3.1-vl
> >>>>> >>>>> > > >> >>>>>>>an
> >>>>> >>>>> > > >> >> >>>>>:/
> >>>>> >>>>> > > >> >> >> >>>/4
> >>>>> >>>>> > > >> >> >> >> >3
> >>>>> >>>>> > > >> >> >> >> >of type Guest from the nics passed on vm
> start. The
> >>>>> >>>>> nic will
> >>>>> >>>>> > > be
> >>>>> >>>>> > > >> >> >>plugged
> >>>>> >>>>> > > >> >> >> >> >later
> >>>>> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,773 DEBUG
> >>>>> >>>>> > > >> >> >> >>
> >[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
> >>>>> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af)
> >>>>> Removing
> >>>>> >>>>> nic
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>NicProfile[1095-801-cd9fd29a-0573-4715-8742-00ecb9f82c9d-10.0.4.1-vl
> >>>>> >>>>> > > >> >>>>>>>an
> >>>>> >>>>> > > >> >> >>>>>:/
> >>>>> >>>>> > > >> >> >> >>>/3
> >>>>> >>>>> > > >> >> >> >> >004
> >>>>> >>>>> > > >> >> >> >> >of type Guest from the nics passed on vm
> start. The
> >>>>> >>>>> nic will
> >>>>> >>>>> > > be
> >>>>> >>>>> > > >> >> >>plugged
> >>>>> >>>>> > > >> >> >> >> >later
> >>>>> >>>>> > > >> >> >> >> >2014-05-30 13:56:23,773 DEBUG
> >>>>> >>>>> > > >> >> >> >>
> >[c.c.n.r.VpcVirtualNetworkApplianceManagerImpl]
> >>>>> >>>>> > > >> >> >> >> >(Job-Executor-77:ctx-ec3d358e ctx-f35b12af)
> >>>>> Removing
> >>>>> >>>>> nic
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>>
> >>>>>>>NicProfile[1095-801-cd9fd29a-0573-4715-8742-00ecb9f82c9d-10.0.4.1-vl
> >>>>> >>>>> > > >> >>>>>>>an
> >>>>> >>>>> > > >> >> >>>>>:/
> >>>>> >>>>> > > >> >> >> >>>/3
> >>>>> >>>>> > > >> >> >> >> >004
> >>>>> >>>>> > > >> >> >> >> >of type Guest from the nics passed on vm start
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >Thanks,
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >On 30 May 2014 13:54, Joris van Lieshout
> >>>>> >>>>> > > >> >> >> >><JvanLieshout@schubergphilis.com
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >wrote:
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> Hi Andrija,
> >>>>> >>>>> > > >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> Just the start of the VR should be
> sufficient.
> >>>>> >>>>> > > >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> Kind regards,
> >>>>> >>>>> > > >> >> >> >> >> Joris van Lieshout
> >>>>> >>>>> > > >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> Schuberg Philis
> >>>>> >>>>> > > >> >> >> >> >> Boeingavenue 271
> >>>>> >>>>> > > >> >> >> >> >> 1119 PD  Schiphol-Rijk
> >>>>> >>>>> > > >> >> >> >> >> schubergphilis.com
> >>>>> >>>>> > > >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> +31 20-7506672
> >>>>> >>>>> > > >> >> >> >> >> +31 6-51428188
> >>>>> >>>>> > > >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> On 30/05/14 13:48, "Andrija Panic" <
> >>>>> >>>>> > andrija.panic@gmail.com
> >>>>> >>>>> > > >
> >>>>> >>>>> > > >> >> >>wrote:
> >>>>> >>>>> > > >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >Hi Joris,
> >>>>> >>>>> > > >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >just to be sure - you want me to capture
> the log
> >>>>> >>>>> from the
> >>>>> >>>>> > > >> >>moment
> >>>>> >>>>> > > >> >> >>I
> >>>>> >>>>> > > >> >> >> >> >>reboot
> >>>>> >>>>> > > >> >> >> >> >> >router - or you want me to stop it, then
> start
> >>>>> >>>>> capturing
> >>>>> >>>>> > > log,
> >>>>> >>>>> > > >> >>and
> >>>>> >>>>> > > >> >> >> >> >>start it
> >>>>> >>>>> > > >> >> >> >> >> >(and continue capture untill ethnull errors
> >>>>> inside
> >>>>> >>>>> VR) ?
> >>>>> >>>>> > > >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >Thanks,
> >>>>> >>>>> > > >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >On 30 May 2014 13:39, Joris van Lieshout
> >>>>> >>>>> > > >> >> >> >> >><JvanLieshout@schubergphilis.com
> >>>>> >>>>> > > >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >wrote:
> >>>>> >>>>> > > >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> Hi Andrija,
> >>>>> >>>>> > > >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> Thanks for the answers. In deed your
> >>>>> situation is
> >>>>> >>>>> > > different
> >>>>> >>>>> > > >> >>so
> >>>>> >>>>> > > >> >> >> >> >>PV/HVM is
> >>>>> >>>>> > > >> >> >> >> >> >> not the issue.
> >>>>> >>>>> > > >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> When reading back the log output you have
> >>>>> >>>>> provided I
> >>>>> >>>>> > > noted
> >>>>> >>>>> > > >> >>that
> >>>>> >>>>> > > >> >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >>VR
> >>>>> >>>>> > > >> >> >> >> >> >> messages log indicates that it's waiting
> for
> >>>>> >>>>> ethnull to
> >>>>> >>>>> > > be
> >>>>> >>>>> > > >> >>up.
> >>>>> >>>>> > > >> >> >> >>This
> >>>>> >>>>> > > >> >> >> >> >> >>raises
> >>>>> >>>>> > > >> >> >> >> >> >> the question where null was introduced
> >>>>> instead of
> >>>>> >>>>> 1.
> >>>>> >>>>> > The
> >>>>> >>>>> > > >> ACS
> >>>>> >>>>> > > >> >> >> >> >>management
> >>>>> >>>>> > > >> >> >> >> >> >> log output you send was, what I think,
> later
> >>>>> down
> >>>>> >>>>> the
> >>>>> >>>>> > > road
> >>>>> >>>>> > > >> >> >>where
> >>>>> >>>>> > > >> >> >> >>ACS
> >>>>> >>>>> > > >> >> >> >> >> >>gives
> >>>>> >>>>> > > >> >> >> >> >> >> up trying to wait for the VR to come up.
> If
> >>>>> you
> >>>>> >>>>> would
> >>>>> >>>>> > > >> >>capture
> >>>>> >>>>> > > >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >> >> job-executor in the management log from
> >>>>> >>>>> startCommand
> >>>>> >>>>> > till
> >>>>> >>>>> > > >> >>the
> >>>>> >>>>> > > >> >> >> >> >>exception,
> >>>>> >>>>> > > >> >> >> >> >> >> do you see anywhere a mention of
> ethnull? You
> >>>>> >>>>> might
> >>>>> >>>>> > need
> >>>>> >>>>> > > to
> >>>>> >>>>> > > >> >> >>reed
> >>>>> >>>>> > > >> >> >> >>into
> >>>>> >>>>> > > >> >> >> >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >> >> DirectAgent executing the startCommand to
> >>>>> find a
> >>>>> >>>>> clue.
> >>>>> >>>>> > > The
> >>>>> >>>>> > > >> >> >>thing
> >>>>> >>>>> > > >> >> >> >>is
> >>>>> >>>>> > > >> >> >> >> >> >>that I
> >>>>> >>>>> > > >> >> >> >> >> >> only have experience with XS based
> >>>>> environment so
> >>>>> >>>>> I
> >>>>> >>>>> > > cannot
> >>>>> >>>>> > > >> >> >>point
> >>>>> >>>>> > > >> >> >> >>you
> >>>>> >>>>> > > >> >> >> >> >>to
> >>>>> >>>>> > > >> >> >> >> >> >> the exact output to look for. On XS, at
> >>>>> least, it
> >>>>> >>>>> is
> >>>>> >>>>> > > >> >> >> >> >> >> "[c.c.h.x.r.CitrixResourceBase]
> >>>>> >>>>> > > >> >>(DirectAgent-351:ctx-4a51bb9e)
> >>>>> >>>>> > > >> >> >> >> >>Created a
> >>>>> >>>>> > > >> >> >> >> >> >> vif e4c362bd-764b-f651-dc9a-1abd5cb33c43
> on 1"
> >>>>> >>>>> > > >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> Kind regards,
> >>>>> >>>>> > > >> >> >> >> >> >> Joris van Lieshout
> >>>>> >>>>> > > >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> Schuberg Philis
> >>>>> >>>>> > > >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> On 30/05/14 10:48, "Andrija Panic" <
> >>>>> >>>>> > > >> andrija.panic@gmail.com>
> >>>>> >>>>> > > >> >> >> >>wrote:
> >>>>> >>>>> > > >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >Hi Deen,
> >>>>> >>>>> > > >> >> >> >> >> >> >no, in DB there is field "vlan_id" with
> value
> >>>>> >>>>> > > "untagged" -
> >>>>> >>>>> > > >> >> >>that
> >>>>> >>>>> > > >> >> >> >> >> >> >"vlan://untagged" is shown from ACS
> gui, and
> >>>>> is
> >>>>> >>>>> used
> >>>>> >>>>> > in
> >>>>> >>>>> > > >> API
> >>>>> >>>>> > > >> >> >>call
> >>>>> >>>>> > > >> >> >> >>(or
> >>>>> >>>>> > > >> >> >> >> >> >> >better
> >>>>> >>>>> > > >> >> >> >> >> >> >said commands that are seen in
> management
> >>>>> server
> >>>>> >>>>> > logs).
> >>>>> >>>>> > > >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >Best,
> >>>>> >>>>> > > >> >> >> >> >> >> >Andrija
> >>>>> >>>>> > > >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >On 30 May 2014 10:37, Daan Hoogland
> >>>>> >>>>> > > >> >><daan.hoogland@gmail.com>
> >>>>> >>>>> > > >> >> >> >> wrote:
> >>>>> >>>>> > > >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> Andrija,
> >>>>> >>>>> > > >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> Do not just assign a second net
> vlan://500
> >>>>> You
> >>>>> >>>>> have
> >>>>> >>>>> > > one
> >>>>> >>>>> > > >> >>like
> >>>>> >>>>> > > >> >> >> >>that
> >>>>> >>>>> > > >> >> >> >> >>and
> >>>>> >>>>> > > >> >> >> >> >> >> >> you don't want conflicting nets using
> the
> >>>>> same
> >>>>> >>>>> > vlan. I
> >>>>> >>>>> > > >> am
> >>>>> >>>>> > > >> >> >> >> >>wondering
> >>>>> >>>>> > > >> >> >> >> >> >> >> why 'untagged' comes out as
> >>>>> 'vlan://untagged'.
> >>>>> >>>>> I
> >>>>> >>>>> > think
> >>>>> >>>>> > > >> >>that
> >>>>> >>>>> > > >> >> >>is
> >>>>> >>>>> > > >> >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >> >> >> bug. Did you find the string
> >>>>> 'vlan://untagged'
> >>>>> >>>>> in
> >>>>> >>>>> > your
> >>>>> >>>>> > > >> >>db?
> >>>>> >>>>> > > >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> On Fri, May 30, 2014 at 10:20 AM,
> Andrija
> >>>>> Panic
> >>>>> >>>>> > > >> >> >> >> >> >> >><andrija.panic@gmail.com>
> >>>>> >>>>> > > >> >> >> >> >> >> >> wrote:
> >>>>> >>>>> > > >> >> >> >> >> >> >> > Hi Joris,
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > thank you for taking time to
> address this
> >>>>> >>>>> issue :)
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > So...:
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > - I'm on KVM (stock CentOS 6.2
> patched by
> >>>>> >>>>> Inktank
> >>>>> >>>>> > > for
> >>>>> >>>>> > > >> >>CEPH
> >>>>> >>>>> > > >> >> >> >> >> >>support),
> >>>>> >>>>> > > >> >> >> >> >> >> >>OS
> >>>>> >>>>> > > >> >> >> >> >> >> >> is
> >>>>> >>>>> > > >> >> >> >> >> >> >> > Centos 6.5, libvirt 1.2.3 compiled.
> >>>>> >>>>> > > >> >> >> >> >> >> >> > - ACS 4.3 having problems, ACS
> 4.2.1 was
> >>>>> fine
> >>>>> >>>>> > > >> >> >> >> >> >> >> > - not XS, so I guess no answers for
> this
> >>>>> >>>>> part :)
> >>>>> >>>>> > > >> >> >> >> >> >> >> > - guest_os_id is 184 = Debian 7 x64
> >>>>> >>>>> > > >> >> >> >> >> >> >> > - SVM = systemvm-kvm-4.3 = os type
> 184 =
> >>>>> >>>>> Debian 7
> >>>>> >>>>> > > x64
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > This worked previously on 4.2.1 =
> >>>>> template
> >>>>> >>>>> was
> >>>>> >>>>> > > >> ofcourse
> >>>>> >>>>> > > >> >> >> >> >> >> >>systemvm-kvm-4.2
> >>>>> >>>>> > > >> >> >> >> >> >> >> -
> >>>>> >>>>> > > >> >> >> >> >> >> >> > but that was also Debian 7 x64
> type... so
> >>>>> >>>>> this
> >>>>> >>>>> > > should
> >>>>> >>>>> > > >> >>not
> >>>>> >>>>> > > >> >> >>be
> >>>>> >>>>> > > >> >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >> >> >>issues
> >>>>> >>>>> > > >> >> >> >> >> >> >> > (guest not supported by host...)
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > The only thing that might be out of
> >>>>> >>>>> "standard" =
> >>>>> >>>>> > all
> >>>>> >>>>> > > >> >>SVMs
> >>>>> >>>>> > > >> >> >> >>are on
> >>>>> >>>>> > > >> >> >> >> >> >>CEPH
> >>>>> >>>>> > > >> >> >> >> >> >> >>-
> >>>>> >>>>> > > >> >> >> >> >> >> >> > there are official docs on altering
> >>>>> database
> >>>>> >>>>> to
> >>>>> >>>>> > make
> >>>>> >>>>> > > >> >>some
> >>>>> >>>>> > > >> >> >>new
> >>>>> >>>>> > > >> >> >> >> >> >>System
> >>>>> >>>>> > > >> >> >> >> >> >> >> > Offering as default for SSVM and
> CPVM -
> >>>>> what
> >>>>> >>>>> I
> >>>>> >>>>> > did,
> >>>>> >>>>> > > I
> >>>>> >>>>> > > >> >>also
> >>>>> >>>>> > > >> >> >> >>have
> >>>>> >>>>> > > >> >> >> >> >> >>done
> >>>>> >>>>> > > >> >> >> >> >> >> >>same
> >>>>> >>>>> > > >> >> >> >> >> >> >> > config in DB, to make VR use another
> >>>>> System
> >>>>> >>>>> > Offering
> >>>>> >>>>> > > >> as
> >>>>> >>>>> > > >> >> >> >>default
> >>>>> >>>>> > > >> >> >> >> >>-
> >>>>> >>>>> > > >> >> >> >> >> >> >>which
> >>>>> >>>>> > > >> >> >> >> >> >> >> is
> >>>>> >>>>> > > >> >> >> >> >> >> >> > NOT explained in the docs - you
> could use
> >>>>> >>>>> "Change
> >>>>> >>>>> > > >> >> >> >>Offering..."
> >>>>> >>>>> > > >> >> >> >> >> >>button
> >>>>> >>>>> > > >> >> >> >> >> >> >>on
> >>>>> >>>>> > > >> >> >> >> >> >> >> > exiting, shutdown VR to change it
> per
> >>>>> docs...
> >>>>> >>>>> > > >> >> >> >> >> >> >> > But still this worked all fine on
> >>>>> 4.2.1...
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > - regarding
> /var/cache/cloud/cmdline the
> >>>>> >>>>> content
> >>>>> >>>>> > is
> >>>>> >>>>> > > >> >> >>folowing
> >>>>> >>>>> > > >> >> >> >>at
> >>>>> >>>>> > > >> >> >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >> >> >> moment
> >>>>> >>>>> > > >> >> >> >> >> >> >> > root@r-801-VM:~# cat
> >>>>> >>>>> /var/cache/cloud/cmdline
> >>>>> >>>>> > > >> >> >> >> >> >> >> > vpccidr=10.0.0.0/8
> >>>>> domain=cscloud.internal
> >>>>> >>>>> > > >> dns1=8.8.8.8
> >>>>> >>>>> > > >> >> >> dns2=
> >>>>> >>>>> > > >> >> >> >> >> >> >> template=domP
> >>>>> >>>>> > > >> >> >> >> >> >> >> > name=r-801-VM eth0ip=169.254.0.75
> >>>>> >>>>> > > eth0mask=255.255.0.0
> >>>>> >>>>> > > >> >> >> >> >> >>type=vpcrouter
> >>>>> >>>>> > > >> >> >> >> >> >> >> > disable_rp_filter=true
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > Also please note that only eth1
> does not
> >>>>> >>>>> have IP
> >>>>> >>>>> > > info,
> >>>>> >>>>> > > >> >> >>eth0
> >>>>> >>>>> > > >> >> >> >> >> >>(control
> >>>>> >>>>> > > >> >> >> >> >> >> >> > 169.xxx) and all other eh2 and up
> that
> >>>>> are
> >>>>> >>>>> used
> >>>>> >>>>> > for
> >>>>> >>>>> > > >> >>Tiers
> >>>>> >>>>> > > >> >> >> >>get IP
> >>>>> >>>>> > > >> >> >> >> >> >>info
> >>>>> >>>>> > > >> >> >> >> >> >> >> fine.
> >>>>> >>>>> > > >> >> >> >> >> >> >> > I could also manually add IP for
> eth1
> >>>>> >>>>> (public NIC)
> >>>>> >>>>> > > and
> >>>>> >>>>> > > >> >> >>start
> >>>>> >>>>> > > >> >> >> >> >>ifup
> >>>>> >>>>> > > >> >> >> >> >> >> >>eth1 -
> >>>>> >>>>> > > >> >> >> >> >> >> >> > and it works fine, but adding new
> IP Port
> >>>>> >>>>> > Forwarding
> >>>>> >>>>> > > >> >>etc
> >>>>> >>>>> > > >> >> >>does
> >>>>> >>>>> > > >> >> >> >> >>not
> >>>>> >>>>> > > >> >> >> >> >> >> >>work...
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > Daan or somebody said it could be
> >>>>> realted to
> >>>>> >>>>> my
> >>>>> >>>>> > > >> >>"Public"
> >>>>> >>>>> > > >> >> >> >>network
> >>>>> >>>>> > > >> >> >> >> >> >>(in
> >>>>> >>>>> > > >> >> >> >> >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >> >> >> > Zones, Physical Network, eth1
> listing)
> >>>>> is NOT
> >>>>> >>>>> > tagged
> >>>>> >>>>> > > >> >> >> >> >> >> >>(vlan://untagged)...
> >>>>> >>>>> > > >> >> >> >> >> >> >> > Interestingly the only VR that does
> work
> >>>>> >>>>> fine is
> >>>>> >>>>> > the
> >>>>> >>>>> > > >> VR
> >>>>> >>>>> > > >> >> >>used
> >>>>> >>>>> > > >> >> >> >>in
> >>>>> >>>>> > > >> >> >> >> >> >>Shared
> >>>>> >>>>> > > >> >> >> >> >> >> >> > network, but that VR is using IP
> from
> >>>>> Guest
> >>>>> >>>>> IP
> >>>>> >>>>> > range
> >>>>> >>>>> > > >> >>(also
> >>>>> >>>>> > > >> >> >> >> >> >>efectively
> >>>>> >>>>> > > >> >> >> >> >> >> >> > public IPs but on vlan 500)
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > I was instructed to try to change
> Public
> >>>>> IP
> >>>>> >>>>> range
> >>>>> >>>>> > > from
> >>>>> >>>>> > > >> >> >> >>untagged
> >>>>> >>>>> > > >> >> >> >> >>to
> >>>>> >>>>> > > >> >> >> >> >> >> >>vlan
> >>>>> >>>>> > > >> >> >> >> >> >> >> > 500, but I'm not sure how to do
> this, if
> >>>>> >>>>> there is
> >>>>> >>>>> > > any
> >>>>> >>>>> > > >> >>way
> >>>>> >>>>> > > >> >> >>at
> >>>>> >>>>> > > >> >> >> >>all
> >>>>> >>>>> > > >> >> >> >> >> >> >>(editing
> >>>>> >>>>> > > >> >> >> >> >> >> >> > "vlan" table and changing to vlan
> 500
> >>>>> does
> >>>>> >>>>> not
> >>>>> >>>>> > work,
> >>>>> >>>>> > > >> >>after
> >>>>> >>>>> > > >> >> >> >> >> >>rebooting
> >>>>> >>>>> > > >> >> >> >> >> >> >>VR
> >>>>> >>>>> > > >> >> >> >> >> >> >> > from ACS gui).
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > :)
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > So, not sure what is roughly
> expected
> >>>>> date
> >>>>> >>>>> for
> >>>>> >>>>> > 4.4,
> >>>>> >>>>> > > >> but
> >>>>> >>>>> > > >> >> >>right
> >>>>> >>>>> > > >> >> >> >> >>now,
> >>>>> >>>>> > > >> >> >> >> >> >>I'm
> >>>>> >>>>> > > >> >> >> >> >> >> >> > pretty stuck with a big problem of
> all
> >>>>> VPC
> >>>>> >>>>> not
> >>>>> >>>>> > > >> >> >>operational at
> >>>>> >>>>> > > >> >> >> >> >> >>all...
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > Thanks,
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > On 30 May 2014 08:27, Joris van
> Lieshout
> >>>>> <
> >>>>> >>>>> > > >> >> >> >> >> >> >> JvanLieshout@schubergphilis.com>
> >>>>> >>>>> > > >> >> >> >> >> >> >> > wrote:
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> Hi Andrija,
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> Daan asked me to have a look at
> this as
> >>>>> >>>>> well.
> >>>>> >>>>> > > Looking
> >>>>> >>>>> > > >> >>at
> >>>>> >>>>> > > >> >> >>you
> >>>>> >>>>> > > >> >> >> >> >> >>issue I
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> recall having seen something
> similar.
> >>>>> Back
> >>>>> >>>>> then
> >>>>> >>>>> > > when
> >>>>> >>>>> > > >> >> >> >>upgrading
> >>>>> >>>>> > > >> >> >> >> >> >>4.2.1
> >>>>> >>>>> > > >> >> >> >> >> >> >>to
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> 4.3 I though it had to do with out
> own
> >>>>> >>>>> custom
> >>>>> >>>>> > build
> >>>>> >>>>> > > >> >>svm
> >>>>> >>>>> > > >> >> >> >> >>template.
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> Let me fire off some questions
> before
> >>>>> >>>>> explaining
> >>>>> >>>>> > > what
> >>>>> >>>>> > > >> >>the
> >>>>> >>>>> > > >> >> >> >>cause
> >>>>> >>>>> > > >> >> >> >> >> >>was
> >>>>> >>>>> > > >> >> >> >> >> >> >>in
> >>>>> >>>>> > > >> >> >> >> >> >> >> our
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> case. :)
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> - what hypervisor (and version)
> are you
> >>>>> >>>>> using?
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> - if XS, is the new VR a
> >>>>> para-virtualised
> >>>>> >>>>> > instance
> >>>>> >>>>> > > >> >>(PV)
> >>>>> >>>>> > > >> >> >>or
> >>>>> >>>>> > > >> >> >> >> >> >>hardware
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> assisted (HVM)? Do a "xe
> vm-param-list"
> >>>>> on
> >>>>> >>>>> the VR
> >>>>> >>>>> > > >> uuid
> >>>>> >>>>> > > >> >> >>and
> >>>>> >>>>> > > >> >> >> >> >>check
> >>>>> >>>>> > > >> >> >> >> >> >>that
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> param PV-args is set and
> >>>>> HVM-boot-policy is
> >>>>> >>>>> > unset.
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> - what is the OS type of the VR in
> ACS
> >>>>> >>>>> > (guest_os_id
> >>>>> >>>>> > > >> in
> >>>>> >>>>> > > >> >> >> >> >>vm_instance
> >>>>> >>>>> > > >> >> >> >> >> >> >>table
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> and match with table guest_os)
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> - what is the OS type of the SVM
> >>>>> template?
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> Now for the explaining. :)
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> In our case the OS type of the new
> >>>>> template
> >>>>> >>>>> was
> >>>>> >>>>> > not
> >>>>> >>>>> > > >> >> >> >>supported
> >>>>> >>>>> > > >> >> >> >> >>on
> >>>>> >>>>> > > >> >> >> >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> XenServer version we are running.
> >>>>> Therefore
> >>>>> >>>>> the
> >>>>> >>>>> > VR
> >>>>> >>>>> > > >> was
> >>>>> >>>>> > > >> >> >> >>started
> >>>>> >>>>> > > >> >> >> >> >>by
> >>>>> >>>>> > > >> >> >> >> >> >>XS
> >>>>> >>>>> > > >> >> >> >> >> >> >>as
> >>>>> >>>>> > > >> >> >> >> >> >> >> a
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> HVM guest. System vms on XS rely
> on the
> >>>>> >>>>> arguments
> >>>>> >>>>> > > >> >>passed
> >>>>> >>>>> > > >> >> >>to
> >>>>> >>>>> > > >> >> >> >> >>them
> >>>>> >>>>> > > >> >> >> >> >> >>in
> >>>>> >>>>> > > >> >> >> >> >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> PV-args param (ends up on the
> guest in
> >>>>> >>>>> > > >> >> >> >>/var/cache/cloud/cmdline
> >>>>> >>>>> > > >> >> >> >> >> >> >>which in
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> turn is used by
> cloud-early-config) in
> >>>>> >>>>> order to
> >>>>> >>>>> > > work.
> >>>>> >>>>> > > >> >> >> >>cmdline
> >>>>> >>>>> > > >> >> >> >> >> >> >>contains
> >>>>> >>>>> > > >> >> >> >> >> >> >> the
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> NIC configuration information.
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> So, long story short, if a VR gets
> >>>>> started
> >>>>> >>>>> as a
> >>>>> >>>>> > HVM
> >>>>> >>>>> > > >> it
> >>>>> >>>>> > > >> >> >>will
> >>>>> >>>>> > > >> >> >> >>not
> >>>>> >>>>> > > >> >> >> >> >> >>get
> >>>>> >>>>> > > >> >> >> >> >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> information needed to configure
> it's
> >>>>> NICs.
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> Workaround
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> We corrected the os_type_id in the
> DB
> >>>>> (yes
> >>>>> >>>>> I know
> >>>>> >>>>> > > >> >>editing
> >>>>> >>>>> > > >> >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >>DB
> >>>>> >>>>> > > >> >> >> >> >> >>is
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> something you usually don't want
> but
> >>>>> there
> >>>>> >>>>> is no
> >>>>> >>>>> > > >> other
> >>>>> >>>>> > > >> >> >>way
> >>>>> >>>>> > > >> >> >> >>in
> >>>>> >>>>> > > >> >> >> >> >>this
> >>>>> >>>>> > > >> >> >> >> >> >> >>case)
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> of the existing VR's and of the
> >>>>> >>>>> systemvmtemplate
> >>>>> >>>>> > to
> >>>>> >>>>> > > >> >> >> >>something
> >>>>> >>>>> > > >> >> >> >> >> >> >>supported
> >>>>> >>>>> > > >> >> >> >> >> >> >> by
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> XenServer.
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> Kind regards,
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> Joris van Lieshout
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> Schuberg Philis
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> On 29/05/14 12:18, "Andrija Panic"
> >>>>> >>>>> > > >> >> >><andrija.panic@gmail.com
> >>>>> >>>>> > > >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >>wrote:
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >They are 2 traffic types on 1
> physical
> >>>>> net
> >>>>> >>>>> (that
> >>>>> >>>>> > > is
> >>>>> >>>>> > > >> >>both
> >>>>> >>>>> > > >> >> >> >> >>tagged
> >>>>> >>>>> > > >> >> >> >> >> >>vlan
> >>>>> >>>>> > > >> >> >> >> >> >> >> 500,
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >and untagged packets travel over
> same
> >>>>> KVM
> >>>>> >>>>> > bridge,
> >>>>> >>>>> > > >> and
> >>>>> >>>>> > > >> >> >>over
> >>>>> >>>>> > > >> >> >> >> >>eth1
> >>>>> >>>>> > > >> >> >> >> >> >>to
> >>>>> >>>>> > > >> >> >> >> >> >> >> outside
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >world)...
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >On 29 May 2014 12:04, Daan
> Hoogland
> >>>>> >>>>> > > >> >> >> >><daan.hoogland@gmail.com>
> >>>>> >>>>> > > >> >> >> >> >> >> wrote:
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> Are these two traffic types in
> one
> >>>>> >>>>> physical
> >>>>> >>>>> > net?
> >>>>> >>>>> > > >> or
> >>>>> >>>>> > > >> >> >>two
> >>>>> >>>>> > > >> >> >> >> >> >>physical
> >>>>> >>>>> > > >> >> >> >> >> >> >>nets
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> on the same interface (seems
> wrong).
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> On Thu, May 29, 2014 at 11:35
> AM,
> >>>>> Jayapal
> >>>>> >>>>> > Reddy
> >>>>> >>>>> > > >> >>Uradi
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> <jayapalreddy.uradi@citrix.com>
> >>>>> wrote:
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> > I don't think editing DB
> table will
> >>>>> >>>>> work.
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> > -Jayapal
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> > On 29-May-2014, at 2:52 PM,
> Andrija
> >>>>> >>>>> Panic
> >>>>> >>>>> > > >> >> >> >> >> >> >><andrija.panic@gmail.com
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> wrote:
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >> It's like this:
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >> I have public subnet /24.
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >> half is dedicated for Guest
> >>>>> traffic
> >>>>> >>>>> (vlan
> >>>>> >>>>> > > 500)
> >>>>> >>>>> > > >> >>and
> >>>>> >>>>> > > >> >> >>the
> >>>>> >>>>> > > >> >> >> >> >> >>second
> >>>>> >>>>> > > >> >> >> >> >> >> >> half is
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >> dedicated to Public
> >>>>> traffic/network
> >>>>> >>>>> (no
> >>>>> >>>>> > vlan
> >>>>> >>>>> > > >> >>tags,
> >>>>> >>>>> > > >> >> >> >>that
> >>>>> >>>>> > > >> >> >> >> >>is
> >>>>> >>>>> > > >> >> >> >> >> >> >> untagged
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> packets)
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >> Both vlan500 and untagged
> packets
> >>>>> >>>>> travel
> >>>>> >>>>> > over
> >>>>> >>>>> > > >> >> >>physical
> >>>>> >>>>> > > >> >> >> >> >>eth1
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >>interface on
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >> hypervisors and can reach
> >>>>> Internet.
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >> Thanks,
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >> On 29 May 2014 11:06, Daan
> >>>>> Hoogland
> >>>>> >>>>> > > >> >> >> >> >> >><daan.hoogland@gmail.com>
> >>>>> >>>>> > > >> >> >> >> >> >> >> wrote:
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>> On Thu, May 29, 2014 at
> 10:57 AM,
> >>>>> >>>>> Andrija
> >>>>> >>>>> > > >> >>Panic <
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> andrija.panic@gmail.com>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>> wrote:
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>>> 500
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>> is 500 the vlan of your
> >>>>> guestnetwork
> >>>>> >>>>> or
> >>>>> >>>>> > your
> >>>>> >>>>> > > >> >> >>physical
> >>>>> >>>>> > > >> >> >> >> >> >>network?
> >>>>> >>>>> > > >> >> >> >> >> >> >> You
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>> wouldn't want to have two
> nets
> >>>>> with
> >>>>> >>>>> vlan
> >>>>> >>>>> > > 500!
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>> --
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>> Daan
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >> --
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >> Andrija Panić
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> --------------------------------------
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>  http://admintweets.com
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >>
> >>>>> --------------------------------------
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> --
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >> Daan
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >--
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >Andrija Panić
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >--------------------------------------
> >>>>> >>>>> > > >> >> >> >> >> >> >> >> >  http://admintweets.com
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >--------------------------------------
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > --
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >> > Andrija Panić
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> --------------------------------------
> >>>>> >>>>> > > >> >> >> >> >> >> >> >   http://admintweets.com
> >>>>> >>>>> > > >> >> >> >> >> >> >> >
> --------------------------------------
> >>>>> >>>>> > > >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >> --
> >>>>> >>>>> > > >> >> >> >> >> >> >> Daan
> >>>>> >>>>> > > >> >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >--
> >>>>> >>>>> > > >> >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >> >Andrija Panić
> >>>>> >>>>> > > >> >> >> >> >> >> >--------------------------------------
> >>>>> >>>>> > > >> >> >> >> >> >> >  http://admintweets.com
> >>>>> >>>>> > > >> >> >> >> >> >> >--------------------------------------
> >>>>> >>>>> > > >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >--
> >>>>> >>>>> > > >> >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >> >Andrija Panić
> >>>>> >>>>> > > >> >> >> >> >> >--------------------------------------
> >>>>> >>>>> > > >> >> >> >> >> >  http://admintweets.com
> >>>>> >>>>> > > >> >> >> >> >> >--------------------------------------
> >>>>> >>>>> > > >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >>
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >--
> >>>>> >>>>> > > >> >> >> >> >
> >>>>> >>>>> > > >> >> >> >> >Andrija Panić
> >>>>> >>>>> > > >> >> >> >> >--------------------------------------
> >>>>> >>>>> > > >> >> >> >> >  http://admintweets.com
> >>>>> >>>>> > > >> >> >> >> >--------------------------------------
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >>
> >>>>> >>>>> > > >> >> >> >
> >>>>> >>>>> > > >> >> >> >
> >>>>> >>>>> > > >> >> >> >--
> >>>>> >>>>> > > >> >> >> >
> >>>>> >>>>> > > >> >> >> >Andrija Panić
> >>>>> >>>>> > > >> >> >> >--------------------------------------
> >>>>> >>>>> > > >> >> >> >  http://admintweets.com
> >>>>> >>>>> > > >> >> >> >--------------------------------------
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >>
> >>>>> >>>>> > > >> >> >
> >>>>> >>>>> > > >> >> >
> >>>>> >>>>> > > >> >> >--
> >>>>> >>>>> > > >> >> >
> >>>>> >>>>> > > >> >> >Andrija Panić
> >>>>> >>>>> > > >> >> >--------------------------------------
> >>>>> >>>>> > > >> >> >  http://admintweets.com
> >>>>> >>>>> > > >> >> >--------------------------------------
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >>
> >>>>> >>>>> > > >> >
> >>>>> >>>>> > > >> >
> >>>>> >>>>> > > >> >--
> >>>>> >>>>> > > >> >
> >>>>> >>>>> > > >> >Andrija Panić
> >>>>> >>>>> > > >> >--------------------------------------
> >>>>> >>>>> > > >> >  http://admintweets.com
> >>>>> >>>>> > > >> >--------------------------------------
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >>
> >>>>> >>>>> > > >
> >>>>> >>>>> > > >
> >>>>> >>>>> > > > --
> >>>>> >>>>> > > >
> >>>>> >>>>> > > > Andrija Panić
> >>>>> >>>>> > > > --------------------------------------
> >>>>> >>>>> > > >   http://admintweets.com
> >>>>> >>>>> > > > --------------------------------------
> >>>>> >>>>> > > >
> >>>>> >>>>> > >
> >>>>> >>>>> > >
> >>>>> >>>>> > >
> >>>>> >>>>> > > --
> >>>>> >>>>> > >
> >>>>> >>>>> > > Andrija Panić
> >>>>> >>>>> > > --------------------------------------
> >>>>> >>>>> > >   http://admintweets.com
> >>>>> >>>>> > > --------------------------------------
> >>>>> >>>>> > >
> >>>>> >>>>> >
> >>>>> >>>>>
> >>>>> >>>>>
> >>>>> >>>>>
> >>>>> >>>>> --
> >>>>> >>>>>
> >>>>> >>>>> Andrija Panić
> >>>>> >>>>> --------------------------------------
> >>>>> >>>>>   http://admintweets.com
> >>>>> >>>>> --------------------------------------
> >>>>> >>>>>
> >>>>> >>>>
> >>>>> >>>>
> >>>>> >>>
> >>>>> >>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Daan
> >>>>>
> >>>>
> >>>>
> >>
> >>
> >>
> >> --
> >> Daan
> >
> >
> >
> > --
> > Daan
>
>
>
> --
> Daan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message