cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kurt K <k...@ihnetworks.com>
Subject Re: Templates are not showing
Date Thu, 17 Nov 2016 02:51:06 GMT
Hi Dag,

We have executed the *SSVM health check* systemvm-ssvm-check.sh script 
on SSVM  and its results are pasted below.

====
root@v-2-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh
================================================
First DNS server is 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 48 data bytes
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
WARNING: cannot ping DNS server
route follows
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 XX.XX.XX.XX 0.0.0.0 UG 0 0 0 eth2
8.8.8.8 10.10.48.1 255.255.255.255 UGH 0 0 0 eth1
10.10.48.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
XX.XX.XX.XX 0.0.0.0 255.255.255.240 U 0 0 0 eth2
XX.XX.XX.XX 10.10.48.1 255.255.255.255 UGH 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
================================================
ERROR: DNS not resolving download.cloud.com
resolv.conf follows
nameserver 8.8.8.8
nameserver 8.8.8.8
nameserver 4.2.2.1
root@v-2-VM:~#
====

But we were unable to connect to in/out from SSVM.  There is one private 
IP and public IP on SSVM.  We have checked the firewall and there was no 
reject rules in it.

Any suggestions?

-Kurt

On 11/11/2016 06:22 PM, Dag Sonstebo wrote:
> Hi Kurt,
>
> My original comment stands – it seems you do not have a healthy SSVM at the moment
– until this has been sorted you will not be able to upload templates.
>
> The systemvm-ssvm-check.sh script lives on the SSVM itself – please see https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM,+templates,+Secondary+storage+troubleshooting
for how to access this. When you run the script it will do the basic checks for connectivity
and NFS access and hopefully give you an idea what is wrong. If you can run this and post
up the results we can advise further.
>
> Regards,
> Dag Sonstebo
>
> On 11/11/2016, 12:42, "Kurt K" <kurt@ihnetworks.com> wrote:
>
>      Hi Dag,
>      
>      We have recreated the SSVM, but still no luck :( . Regarding
>      ssvm-check.sh check script,  it was not presented on management server.
>      
>      ===
>      [root@manage1 /]# ls /usr/local/cloud/systemvm-ssvm-check.sh
>      ls: cannot access /usr/local/cloud/systemvm-ssvm-check.sh: No such file
>      or directory
>      ===
>      
>      While registering an ISO, we are getting the following errors.
>      
>      ===
>      2016-11-11 03:27:28,998 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>      (catalina-exec-7:ctx-23ce7527 ctx-47f4a89e) template 202 is not in
>      store:1, type:Image
>      2016-11-11 03:27:29,118 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>      (catalina-exec-7:ctx-23ce7527 ctx-47f4a89e) template 202 is already in
>      store:1, type:Image
>      2016-11-11 03:27:29,257 DEBUG [o.a.c.s.i.BaseImageStoreDriverImpl]
>      (catalina-exec-7:ctx-23ce7527 ctx-47f4a89e) Downloading template to data
>      store 1
>      2016-11-11 03:27:29,263 WARN [c.c.s.d.DownloadMonitorImpl]
>      (catalina-exec-7:ctx-23ce7527 ctx-47f4a89e) There is no secondary
>      storage VM for downloading template to image store
>      nfs://162.x.x.x/export/secondary
>      ===
>      
>      We had tried every possible workarounds, but none of them working. Is
>      this have any relation with NFS version on server? we are using nfs 4.
>      Moreover I did some searches and  some one recommend to set this
>      parameter "secstorage.allowed.internal.sites" value. I have set this to
>      our IP range, still no luck.
>      
>      
>      We have followed the general installation step on this doc
>      http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.6/index.html
>      . Are we miss anything on NFS server setup side? please let us know .
>      Also this time, we have excluded the NFS server and use management
>      server as NFS itself. The same issue out there.
>      
>      Do you have any other specific step or any other instructions for
>      configuring the cloudstack apart from cloudstack doc ?
>      
>      We were totally stuck here.
>      
>      -Kurt
>      
>      On 11/10/2016 09:09 PM, Dag Sonstebo wrote:
>      > Sounds like your SSVM isn’t too healthy – I would first of all delete it,
let CS recreate the new one and see if the template starts downloading thereafter.
>      > If you still have problems afterwards do some troubleshooting on your new SSVM
– e.g. run the /usr/local/cloud/systemvm-ssvm-check.sh script and see if all your connectivity
is in place.
>      >
>      > Regards,
>      > Dag Sonstebo
>      >
>      > On 10/11/2016, 13:46, "Kurt K" <kurt@ihnetworks.com> wrote:
>      >
>      >      Hi Dag,
>      >
>      >       >> Can you clarify what the problem is?
>      >
>      >      Please see the below URLs.
>      >      ===
>      >      http://s1152.photobucket.com/user/vincemd12/media/CloudS1_zps1uz6qwjt.png.html
>      >      http://s1152.photobucket.com/user/vincemd12/media/CloudS_zps59mfxgti.png.html
>      >      ===
>      >
>      >      When we try to create the instance via management UI, there were no
>      >      default templates detected on template selection option. The above
>      >      pasted URLs described the same. 2 system VMs are also running fine. At
>      >      the same time, the management server log captures the below debug errors.
>      >
>      >      ===
>      >      2016-11-10 05:42:10,947 DEBUG [c.c.s.StatsCollector]
>      >      (StatsCollector-3:ctx-81097a24) There is no secondary storage VM for
>      >      secondary storage host nfs://162.x.x.x/export/secondary
>      >      ===
>      >
>      >      Any idea?
>      >
>      >      Regards,
>      >      Kurt
>      >
>      >
>      >      On 11/10/2016 04:27 PM, Dag Sonstebo wrote:
>      >      > Hi Kurt,
>      >      >
>      >      > That looks all healthy, and the following shows your system VM template
+ the standard CentOS 5.6 template being OK:
>      >      >
>      >      > | 3 | routing-3 | SystemVM Template (KVM) | SYSTEM | 2016-11-10 01:48:12|
NULL | KVM | Active |
>      >      > | 4 | centos55-x86_64 | CentOS 5.5(64-bit) no GUI (KVM) | BUILTIN
|2016-11-10 01:48:12 | NULL | KVM | Active |
>      >      >
>      >      > In addition to that – if you have system VMs running you must by
definition have the system VM template downloaded OK.
>      >      >
>      >      > Can you clarify what the problem is?
>      >      >
>      >      > Regards,
>      >      > Dag Sonstebo
>      >      >
>      >      > On 10/11/2016, 10:35, "Kurt K" <kurt@ihnetworks.com> wrote:
>      >      >
>      >      >      Hi Dag,
>      >      >
>      >      >      The mentioned details are pasted below.
>      >      >
>      >      >      =====
>      >      >      [root@nfs1 secondary]# tree .
>      >      >      .
>      >      >      └── template
>      >      >           └── tmpl
>      >      >               └── 1
>      >      >                   └── 3
>      >      >                       ├── 073ecae0-38a0-4a08-87c2-0f5c0f725955.qcow2
>      >      >                       └── template.properties
>      >      >
>      >      >      4 directories, 2 files
>      >      >      ==
>      >      >      mysql> SELECT
>      >      >      id,unique_name,name,type,created,removed,hypervisor_type,state
FROM
>      >      >      cloud.vm_template;
>      >      >      +-----+---------------------+---------------------------------------+---------+---------------------+---------+-----------------+----------+
>      >      >      | id | unique_name | name | type | created | removed | hypervisor_type
|
>      >      >      state |
>      >      >      +-----+---------------------+---------------------------------------+---------+---------------------+---------+-----------------+----------+
>      >      >      | 1 | routing-1 | SystemVM Template (XenServer) | SYSTEM | 2016-11-10
>      >      >      01:48:12 | NULL | XenServer | Active |
>      >      >      | 2 | centos53-x86_64 | CentOS 5.3(64-bit) no GUI (XenServer)
| BUILTIN
>      >      >      | 2016-11-10 01:48:12 | NULL | XenServer | Inactive |
>      >      >      | 3 | routing-3 | SystemVM Template (KVM) | SYSTEM | 2016-11-10
01:48:12
>      >      >      | NULL | KVM | Active |
>      >      >      | 4 | centos55-x86_64 | CentOS 5.5(64-bit) no GUI (KVM) | BUILTIN
|
>      >      >      2016-11-10 01:48:12 | NULL | KVM | Active |
>      >      >      | 5 | centos56-x86_64-xen | CentOS 5.6(64-bit) no GUI (XenServer)
|
>      >      >      BUILTIN | 2016-11-10 01:48:12 | NULL | XenServer | Active |
>      >      >      | 6 | centos64-x64 | CentOS 6.4(64-bit) GUI (Hyperv) | BUILTIN
|
>      >      >      2016-11-10 01:48:13 | NULL | Hyperv | Active |
>      >      >      | 7 | centos53-x64 | CentOS 5.3(64-bit) no GUI (vSphere) | BUILTIN
|
>      >      >      2016-11-10 01:48:13 | NULL | VMware | Active |
>      >      >      | 8 | routing-8 | SystemVM Template (vSphere) | SYSTEM | 2016-11-10
>      >      >      01:48:13 | NULL | VMware | Active |
>      >      >      | 9 | routing-9 | SystemVM Template (HyperV) | SYSTEM | 2016-11-10
>      >      >      01:48:13 | NULL | Hyperv | Active |
>      >      >      | 10 | routing-10 | SystemVM Template (LXC) | SYSTEM | 2016-11-10
>      >      >      01:50:37 | NULL | LXC | Active |
>      >      >      | 200 | vmware-tools.iso | vmware-tools.iso | PERHOST | 2016-11-10
>      >      >      09:53:02 | NULL | VMware | Active |
>      >      >      | 201 | xs-tools.iso | xs-tools.iso | PERHOST | 2016-11-10 09:53:02
|
>      >      >      NULL | XenServer | Active |
>      >      >      +-----+---------------------+---------------------------------------+---------+---------------------+---------+-----------------+----------+
>      >      >      12 rows in set (0.00 sec)
>      >      >      =====
>      >      >
>      >      >       From SystemVMs under the infrastructure, shows 2 VMs running.
One is
>      >      >      console-proxy server and other one is secondary storage VM.
>      >      >
>      >      >      We have connected to the servers via internal IP.
>      >      >
>      >      >      -Kurt
>      >      >
>      >      >
>      >      >      On 11/10/2016 02:51 PM, Dag Sonstebo wrote:
>      >      >      > Hi Kurt,
>      >      >      >
>      >      >      > Can you provide the following:
>      >      >      >
>      >      >      > - A file listing (e.g. a “tree” dump) of the contents
of nfs://162.219.249.166/export/secondary
>      >      >      > - Output of the following from your DB: SELECT id,unique_name,name,type,created,removed,hypervisor_type,state
FROM cloud.vm_template;
>      >      >      >
>      >      >      > Going by the logs it could look like the system VM template
is downloaded OK, but your problem may be that system VMs are unable to start.
>      >      >      >
>      >      >      >
>      >      >      > Regards,
>      >      >      > Dag Sonstebo
>      >      >      >
>      >      >      > On 10/11/2016, 05:54, "Kurt K" <kurt@ihnetworks.com>
wrote:
>      >      >      >
>      >      >      >      Hi Dag,
>      >      >      >
>      >      >      >      We were getting below logs from management server.
>      >      >      >
>      >      >      >
>      >      >      >      ====
>      >      >      >      2016-11-09 21:36:53,096 DEBUG [c.c.a.t.Request]
>      >      >      >      (AgentManager-Handler-4:null) Seq 1-4251679523214458881:
Processing:  {
>      >      >      >      Ans: , MgmtId: 207381009036, via: 1, Ver: v1, Flags:
10,
>      >      >      >      [{"com.cloud.agent.api.ModifyStoragePoolAnswer":{"poolInfo":{"host":"162.x.x.x","localPath":"/mnt//380ea4ee-87df-39eb-92f4-730dfea4655a","hostPath":"/export/primary","poolType":"NetworkFilesystem","capacityBytes":979751665664,"availableBytes":977581113344},"templateInfo":{},"result":true,"wait":0}}]
>      >      >      >      }
>      >      >      >      2016-11-09 21:36:53,451 INFO  [o.a.c.s.i.TemplateServiceImpl]
>      >      >      >      (AgentConnectTaskPool-1:ctx-4e501d7e) Downloading
builtin template
>      >      >      >      centos55-x86_64 to data center: 1
>      >      >      >      2016-11-09 21:36:53,461 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>      >      >      >      (AgentConnectTaskPool-1:ctx-4e501d7e) template 4 is
already in store:1,
>      >      >      >      type:Image
>      >      >      >      2016-11-09 21:36:53,571 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>      >      >      >      (AgentConnectTaskPool-1:ctx-4e501d7e) template 4 is
already in store:1,
>      >      >      >      type:Image
>      >      >      >      2016-11-09 21:36:53,709 DEBUG [o.a.c.s.i.BaseImageStoreDriverImpl]
>      >      >      >      (AgentConnectTaskPool-1:ctx-4e501d7e) Downloading
template to data store 1
>      >      >      >      2016-11-09 21:36:53,720 WARN  [c.c.s.d.DownloadMonitorImpl]
>      >      >      >      (AgentConnectTaskPool-1:ctx-4e501d7e) There is no
secondary storage VM
>      >      >      >      for downloading template to image store
>      >      >      >      nfs://162.219.249.166/export/secondary
>      >      >      >      2016-11-09 21:36:53,725 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl]
>      >      >      >      (AgentConnectTaskPool-1:ctx-4e501d7e) template 4 is
already in store:1,
>      >      >      >      type:Image
>      >      >      >      ====
>      >      >      >
>      >      >      >      -Kurt
>      >      >      >
>      >      >      >      On 11/10/2016 11:05 AM, Kurt K wrote:
>      >      >      >      > Hi Dag,
>      >      >      >      >
>      >      >      >      > We had mounted the NFS share on management server
before executed the
>      >      >      >      > script. Also there was no  /secondary or other
local directory on our
>      >      >      >      > management server. All that presented is NFS
secondary share.
>      >      >      >      >
>      >      >      >      > Can we change any global settings or alter cloud
DB entries for the
>      >      >      >      > working of this template?
>      >      >      >      >
>      >      >      >      > -Kurt
>      >      >      >      >
>      >      >      >      >
>      >      >      >      > On 11/09/2016 07:54 PM, Dag Sonstebo wrote:
>      >      >      >      >> Hi Kurt,
>      >      >      >      >>
>      >      >      >      >> As Simon is hinting at it looks like you
didn’t mount the secondary
>      >      >      >      >> storage NFS share on your management server
before running:
>      >      >      >      >>
>      >      >      >      >> "/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
>      >      >      >      >>
>      >      >      >      >> -m /secondary -u
>      >      >      >      >> http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2
>      >      >      >      >>
>      >      >      >      >> -h kvm -F"
>      >      >      >      >>
>      >      >      >      >> In other words the template downloaded to
the local folder /secondary
>      >      >      >      >> on the management server rather than to a
mounted NFS share.
>      >      >      >      >> If this is the case delete the contents of
your local /secondary
>      >      >      >      >> folder, mount the NFS share in the same folder
and re-run the script
>      >      >      >      >> above.
>      >      >      >      >>
>      >      >      >      >> With regards to downloading from the GUI
– keep in mind you have a
>      >      >      >      >> chicken-and-egg scenario – until you have
first downloaded the system
>      >      >      >      >> VM template with the above script CloudStack
can not create a SSVM,
>      >      >      >      >> hence it will not be able to download any
further templates.
>      >      >      >      >>
>      >      >      >      >> Regards,
>      >      >      >      >> Dag Sonstebo
>      >      >      >      >>
>      >      >      >      >> From: Kurt K <kurt@ihnetworks.com>
>      >      >      >      >> Reply-To: "users@cloudstack.apache.org" <users@cloudstack.apache.org>
>      >      >      >      >> Date: Wednesday, 9 November 2016 at 14:14
>      >      >      >      >> To: "users@cloudstack.apache.org" <users@cloudstack.apache.org>
>      >      >      >      >> Subject: Re: Templates are not showing
>      >      >      >      >>
>      >      >      >      >> Hi Simon,
>      >      >      >      >>
>      >      >      >      >> Thanks for the response.
>      >      >      >      >>
>      >      >      >      >>   >> Is your secondary storage on a
separate NFS server?
>      >      >      >      >>
>      >      >      >      >> Yeah, we have used separate NFS server.
>      >      >      >      >>
>      >      >      >      >>>> Have you added the secondary storage
mount to CloudStack Management?
>      >      >      >      >> We had tried this.  But no luck.
>      >      >      >      >>
>      >      >      >      >> ===
>      >      >      >      >> 162.x.x.x:/export/secondary nfs 913G 2.0G
865G 1% /mnt/secondary
>      >      >      >      >> ===
>      >      >      >      >>
>      >      >      >      >> However, we have tried to download a CentOS
6.5 ISO directly using
>      >      >      >      >> register ISO option from Management UI. There
was no downloading
>      >      >      >      >> status shows and ready status as no. Please
see the below snippets
>      >      >      >      >> and attached screen shots.
>      >      >      >      >>
>      >      >      >      >> ===
>      >      >      >      >> [root@nfs1 3]# pwd;ll
>      >      >      >      >> /export/secondary/template/tmpl/1/3
>      >      >      >      >> total 315396
>      >      >      >      >> -rw-r--r--. 1 root root 322954240 Nov 8 23:45
>      >      >      >      >> 37ad47be-223d-43ff-ae2c-f10548274c95.qcow2
>      >      >      >      >> -rw-r--r--. 1 root root 295 Nov 9 04:30 template.properties
>      >      >      >      >> ===
>      >      >      >      >>
>      >      >      >      >> Any idea about this?
>      >      >      >      >>
>      >      >      >      >> -Kurt
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> On 11/08/2016 10:25 PM, Simon Weller wrote:
>      >      >      >      >>
>      >      >      >      >> Hi Kurt,
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> Is your secondary storage on a separate NFS
server?
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> Have you added the secondary storage mount
to CloudStack Management?
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> If so, have you tried mounting your secondary
storage on a different
>      >      >      >      >> server to make sure the permissions are set
correctly?
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> - Si
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> ________________________________
>      >      >      >      >>
>      >      >      >      >> From: Kurt K <kurt@ihnetworks.com><mailto:kurt@ihnetworks.com>
>      >      >      >      >>
>      >      >      >      >> Sent: Tuesday, November 8, 2016 8:48 AM
>      >      >      >      >>
>      >      >      >      >> To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>
>      >      >      >      >>
>      >      >      >      >> Subject: Fwd: Templates are not showing
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> -------- Forwarded Message --------
>      >      >      >      >>
>      >      >      >      >> Subject:         Templates are not showing
>      >      >      >      >>
>      >      >      >      >> Date:    Tue, 8 Nov 2016 20:09:17 +0530
>      >      >      >      >> Dag.Sonstebo@shapeblue.com
>      >      >      >      >> www.shapeblue.com
>      >      >      >      >> 53 Chandos Place, Covent Garden, London 
WC2N 4HSUK
>      >      >      >      >> @shapeblue
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> From:    Kurt K <kurt@ihnetworks.com><mailto:kurt@ihnetworks.com>
>      >      >      >      >>
>      >      >      >      >> To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> Hi There,
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> We have started to build our own cloud platform
on our organization. Our
>      >      >      >      >>
>      >      >      >      >> deployment is like a small scale arch. We
have used 4 KVM hyper visor
>      >      >      >      >>
>      >      >      >      >> servers and 1 management server and 3 storage
servers. We have
>      >      >      >      >>
>      >      >      >      >> successfully configured all those servers.
But while creating the
>      >      >      >      >>
>      >      >      >      >> instance, there were no templates shown on
management UI. We have
>      >      >      >      >>
>      >      >      >      >> downloaded the templates using this script
>      >      >      >      >>
>      >      >      >      >> "/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> -m /secondary -u
>      >      >      >      >>
>      >      >      >      >> http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> -h kvm -F", it was on secondary partition.
But didn't show up there on
>      >      >      >      >>
>      >      >      >      >> management UI. Is there any other thing need
to be done to show up
>      >      >      >      >>
>      >      >      >      >> templates on UI?
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> Any help or suggestions from your end is
much appreciated.
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> Regards,
>      >      >      >      >>
>      >      >      >      >> Kurt K
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >>
>      >      >      >      >> --
>      >      >      >      >> ~Kurt
>      >      >      >
>      >      >      >
>      >      >      >      --
>      >      >      >      *~Kurt *
>      >      >      >
>      >      >      >
>      >      >      >
>      >      >      > Dag.Sonstebo@shapeblue.com
>      >      >      > www.shapeblue.com
>      >      >      > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>      >      >      > @shapeblue
>      >      >      >
>      >      >      >
>      >      >      >
>      >      >
>      >      >
>      >      >      --
>      >      >      *~Kurt *
>      >      >
>      >      >
>      >      >
>      >      > Dag.Sonstebo@shapeblue.com
>      >      > www.shapeblue.com
>      >      > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>      >      > @shapeblue
>      >      >
>      >      >
>      >      >
>      >
>      >
>      >      --
>      >      *~Kurt *
>      >
>      >
>      >
>      > Dag.Sonstebo@shapeblue.com
>      > www.shapeblue.com
>      > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>      > @shapeblue
>      >
>      >
>      >
>      
>      
>      --
>      *~Kurt *
>      
>
>
>
>
>


-- 
*~Kurt *

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message