Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: pass (athena.apache.org: domain of williamstevens@gmail.com
 designates 209.85.192.174 as permitted sender)
MIME-Version: 1.0
Sender: williamstevens@gmail.com
In-Reply-To: 
 <CAC2pano2uYSwQEeh9n6SrdPHCy5F12s56yxUjmuatiQsL8-uCg@mail.gmail.com>
References: <BDFB04ED-24C1-452A-80F5-D51277213E2F@schubergphilis.com>
	<CADOmhZwktBnO+MNuAOXp1rJBf5G2c6BM1adfqu7ajk9OVTdvqQ@mail.gmail.com>
	<CAC2pano2uYSwQEeh9n6SrdPHCy5F12s56yxUjmuatiQsL8-uCg@mail.gmail.com>
Date: Mon, 1 Dec 2014 17:16:51 -0500
Message-ID: 
 <CAC2panqfSFDVTXqdikWG9bN-_k7yb++EGqGQpfEAw=wJabOO_Q@mail.gmail.com>
Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
From: Will Stevens <wstevens@cloudops.com>
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
Cc: Leo Simons <LSimons@schubergphilis.com>,
	Wilder Rodrigues <WRodrigues@schubergphilis.com>,
 Rohit Yadav <bhaisaab@apache.org>,
	Edison Su <Edison.su@citrix.com>
Content-Type: multipart/alternative; boundary=047d7b44febead422e05092ef731

--047d7b44febead422e05092ef731
Content-Type: text/plain; charset=UTF-8

Alright, so far I have found the following:
https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c

Basically, if the qemu-img version is less than 1.7, we should run the non
'compat' option version and if the version is 1.7 or greater, we should run
the new command with the 'compat' version.

Unfortunately I am not able to find a way to get the qemu-img version from
the command line.

I am looking to basically add a conditional to try running with the compat
option and if that fails, then run it without the compat option.

Basically, I would be replacing this:

qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
"${appliance_build_name}-kvm.qcow2"

With this:

set +e
qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
"${appliance_build_name}-kvm.qcow2"
local qemuresult=$?
set -e
if [ ${qemuresult} != 0 ]; then
  log INFO "qemu-img convert failed, trying without compat option"
  qemu-img convert -f raw -c -O qcow2 raw.img
"${appliance_build_name}-kvm.qcow2"
fi

What do you guys think?  Is this a good enough solution?  If you guys agree
I will implement it in master and make sure it works, then we can merge the
change back to 4.5 to fix that branch as well.

Let me know if you have issues with this approach...

Cheers,

Will


*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens <wstevens@cloudops.com> wrote:

> Edison,
> According to this page the default compat option is 0.10:
> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
>
> Did you find that to not be the case and is that why you had to add the
> compat option?
>
> BTW, in an attempt to get the master system vms building again, I
> committed a change to master to remove the compat option.  We now have
> master system vms building correctly, but Rohit rightly pointed out that I
> had basically reverted your change.
>
> I created a Jira ticket for this issue:
> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
>
> I have not reverted my change in master at this point, so it is building
> right now, but I also did not make the change to 4.5, so that branch is
> currently failing to build system vms.
>
> I will see if I can track down in which version of qemu the compat option
> was added so we can add some intelligent logic around this command.
>
> Will
>
>
>
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Thu, Nov 20, 2014 at 1:07 PM, edison su <sudison@gmail.com> wrote:
>
>> Hi Leo,
>> Our new internal build system are using ubuntu 14.04 or something like
>> that, which has qemu 1.x installed by default, that's why I added the
>> "compat" option in the build script, otherwise, the image build by
>> qemu 1.x, won't work on RHEL 6.x.
>> The fix would be, in the build script, check the version of qemu-img,
>> if it's 0.x, then don't add "compat" option. There are a lot of people
>> still using RHEL 6.x as KVM hypervisor, we have to make sure the image
>> we build can still work on these machines.
>>
>> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <LSimons@schubergphilis.com>
>> wrote:
>> > Hey Edison, all,
>> >
>> >
>> https://github.com/apache/cloudstack/commit/05bec59c1498dbcfb8a1089c86855fd3b433ea58
>> >
>> > breaks our internal build with
>> >
>> >     + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
>> >       systemvmtemplate-systemvm-persistent-config-4.5.0.124-kvm.qcow2
>> >     Unknown option 'compat'
>> >     qemu-img: Invalid options for file format 'qcow2'.
>> >
>> > This is on CentOS release 6.6, which has
>> qemu-img-0.15.0-1.el6.rfx.x86_64.
>> >
>> > Based on
>> >   http://wiki.qemu.org/OlderNews
>> > that seems like it is a really old qemu (august 2011).
>> >
>> > I'm guessing you have a newer OS / newer qemu? Can you please let me
>> know what OS, OS version and qemu(-img) version you are using?
>> >
>> > Also, does anyone know if there some minimum version of qemu-img that
>> should be used / cloudstack assumes? Is 0.15 still ok to do an acceptable
>> image conversion with? (we currently don't have any kvm use ourselves, but,
>> I'd like for our build infra to produce useful kvm images nonetheless).
>> >
>> > According to
>> >   http://wiki.qemu.org/ChangeLog/1.1
>> > the -o compat switch was introduced in 1.1.
>> >
>> > According to
>> >
>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
>> > the default format was changed from 0.10 to 1.1 in qemu 1.7 and onwards.
>> >
>> > The libvirt people
>> >   https://bugzilla.redhat.com/show_bug.cgi?id=997977
>> > say they pass -o compat when qemu supports it (so when v >= 1.1 I
>> imagine).
>> >
>> > I think we should do the same in the build script and I'll make a patch.
>> >
>> > But, should we publish newer-format images too? According to
>> >   http://wiki.qemu.org/Features/Qcow3
>> > the new format is much better so I imagine qcow/kvm users will really
>> appreciate the newer formats.
>> >
>> > Thoughts?
>> >
>> >
>> > cheers,
>> >
>> >
>> > Leo
>> >
>>
>>
>>
>> --
>> Best Regards,
>> Edison
>>
>
>

--047d7b44febead422e05092ef731--