Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
MIME-Version: 1.0
Sender: williamstevens@gmail.com
In-Reply-To: <AA9FDF7E-81FB-4A0F-BC8F-25CD6424A594@shapeblue.com>
References: <AA9FDF7E-81FB-4A0F-BC8F-25CD6424A594@shapeblue.com>
From: Will Stevens <wstevens@cloudops.com>
Date: Tue, 2 May 2017 08:35:56 -0400
Message-ID: <CAC2panpVGPoKGkLPk1+QJ1=6mQft9E=fs=yZtE4vz0U9Zh-ROw@mail.gmail.com>
Subject: Re: ovf file parser
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
Content-Type: multipart/alternative; boundary=94eb2c03863e0b15b6054e89c864
archived-at: Tue, 02 May 2017 12:36:03 -0000

--94eb2c03863e0b15b6054e89c864
Content-Type: text/plain; charset=UTF-8

Hey Abhinandan,
First, can you give us a bit more context regarding what you are doing so
we can highlight potential areas to watch out for?  I have done some OVF
parsing/modification and there are a bunch of gotchas to be aware of.  I
will try to outline some of the ones I found.  I have not tried to use the
vim25.jar, so I can't really help on that front.

In my use case, I was exporting VMs via the ovftool from a source VMware
environment, and I was migrating them to an ACS managed VMware
environment.  In doing so, I also wanted to support VMs with multiple disks
using a Root volume and multiple Data volumes, as well as change the nic
type (vmxnet3), assign static IPs, etc...  I have not had the time to open
source my migration tool, but it is on my todo list.

My general flow was:
- Export the VM with ovftool
- Extract the resulting OVA into its parts (OVF, VMDKs, Manifest)
- Duplicate the OVF file, once per VMDK
- Modify a OVF file to be specific for each of the VMDKs (one OVF per VMDK)
- Take each VMDK and the corresponding OVF and recompress them back into an
OVA
- Treat the first OVA as a template and the rest as data disks

My initial (naive) approach was to just treat the OVF as a well behaved XML
file and use standard XML libs (in my case in Python) to parse and
manipulate the OVF file.  This approach had a few pitfalls which I will
outline here.

VMware is VERY picky about the format of the OVF file, if the file is not
perfect, VMware won't import it (or at least the VM won't launch).  There
were two main items which caused me issues.

a) The <Envelope> tag MUST have all of the namespace definitions even if
they are not used in the file.  This is something that most XML parsers are
confused by.  Most XML parsers will only include the namespaces used in the
file when the file is saved.  I had to ensure that the resulting OVF files
had all of the original namespace definitions for the file to import
correctly.  If I remember correctly, they even had to be in the right
order.  I did this by changing the resulting file after saving it with the
XML lib.

b) VMware uses namespaces which actually collide with each other.  For
example, both the default namespace and the 'ovf' namespace share the same
URL.  Again, XML libraries don't understand this, so I had to manage that
manually.  Luckily, the way VMware handles these namespaces is relatively
consistent, so I was able to find a workaround.  Basically, the default
namespace will apply to all of the elements, and the 'ovf' namespace will
be applied only in the attributes.  Because of this I was able to just use
the 'ovf' namespace and then after exporting the file, I did a find replace
from '<ovf:' and '</ovf:' to '<' and '</' respectively.

Those are the main gotchas which I encountered.

I put the OVA Split function I wrote into a Gist [1] (for now) for your
reference in case reviewing the code is helpful.  I was under a lot of time
pressure when building this tool, so I have a bunch of cleanup to do before
I release it as open source, but I can rush it out and clean it up after
release if you are solving the same(ish) problem and my code will be useful.

[1] https://gist.github.com/swill/f6b54762ffcce85772535a490a9c8cbe

I hope this is helpful in your case.

Cheers,

*Will STEVENS*
Lead Developer

<https://goo.gl/NYZ8KK>

On Tue, May 2, 2017 at 3:49 AM, Abhinandan Prateek <
abhinandan.prateek@shapeblue.com> wrote:

> Hello,
>
> I am looking at vim25.jar to put together ovftool like functionality
> specially around parsing and generating OVF files. vim25.jar  is already
> included as non-oss dependency and used by vmware hypervisor plugin. I see
> that some OVF parsing capabilities are present in this jar, but it seems to
> be tied to host connection/context. Can anyone who has used this can tell
> me if I can use it as a standalone OVF manipulation api any pointer to good
> resource on that will be nice.
>
> Regards,
> -abhi
>
>
>
> abhinandan.prateek@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

--94eb2c03863e0b15b6054e89c864--