cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wido den Hollander <>
Subject Re: RBD snapshot format
Date Sun, 01 Feb 2015 06:54:43 GMT
Hash: SHA1

On 01/30/2015 04:43 PM, Logan Barfield wrote:
> Hi Wido,
> I didn't see the format information for snapshots stored in the 
> database, so I'm assuming it references the associated volume
> format.

Yes, it is.

> I assume the cleanest way to get around that would be to have a 
> separate format field for snapshots, but that wouldn't really make 
> sense just for this one use case.  Another way would be to just
> have the restore (create template from snapshot, etc.) function
> always assume QCOW2 for RBD snapshots, but that's dirty, and would
> cause problems for existing snapshots.

That would be dirty, since I don't think we can actually know if a
snapshpot was from a RBD volume when it's on SS.

Maybe qemu-img can detect the input format? That way we don't have to
specify it.

> It seems like the best method would be to have the restore
> function look at the actual file to get the format information
> instead of pulling it from the database, though I'm not sure if
> that's even possible.

I think that would be the best option indeed.

> Does anyone else have any thoughts or suggestions?
> Thank You,
> Logan Barfield Tranquil Hosting
> On Thu, Jan 29, 2015 at 4:21 PM, Wido den Hollander
> <> wrote:
> On 01/29/2015 10:05 PM, Logan Barfield wrote:
>>>> Hi Wido,
>>>> The relevant code for this is in:
>>>> I saw that the destination format is set to QCOW2 for 
>>>> "createTemplateFromVolume", but in "backupSnapshot" it's set
>>>> to use the format of the source image 
>>>> "destFile.setFormat(snapshotDisk.getFormat());"
>>>> I believe just changing the "backupSnapshot" code to the use
>>>> QCOW2 "destFile.setFormat(PhysicalDiskFormat.QCOW2);" will
>>>> work fine. I can submit a patch for that, but I wanted to
>>>> make sure there wasn't some other reason for the RAW output
>>>> first.
> No, that won't work. Because the management server will then still 
> think it's in RAW format and store that in the database.
> It has been a long time since I looked at this. I know I did some
> work in this area and made some patches.
> Wido
>>>> Thank You,
>>>> Logan Barfield Tranquil Hosting
>>>> On Thu, Jan 29, 2015 at 3:52 PM, Wido den Hollander 
>>>> <> wrote:
>>>> On 01/29/2015 05:16 PM, Logan Barfield wrote:
>>>>>>> I was doing some testing earlier this week on a KVM
>>>>>>> cluster, and noticed that when using S3 for secondary
>>>>>>> storage snapshots on RBD primary storage take a lot
>>>>>>> longer than they do with NFS primary storage.
>>>>>>> I think this is due to the NFS snapshots being 
>>>>>>> output/uploaded in QCOW2 format, while RBD snapshots
>>>>>>> are output in RAW format.  It appears that even when
>>>>>>> using sparse RAW files they are uploaded to S3 with the
>>>>>>> 'allocated' size instead of the sparse size.
>>>> The problem here lies in the code in CloudStack. Since the
>>>> source volume is in RAW format, the snapshot also becomes
>>>> QCOW2.
>>>> I remember that I wrote some patches for this, you might want
>>>> to look in the Git history.
>>>>>>> My question is: Is there a good reason for RBD
>>>>>>> snapshots to be output as RAW instead of QCOW2?  It
>>>>>>> appears that QCOW2 templates are automatically
>>>>>>> converted when deploying them as RBD, so that shouldn't
>>>>>>> be a problem.  The only downside I can think of would
>>>>>>> be having to convert the RAW RBD data to QCOW2 as an
>>>>>>> extra step when creating snapshots, but even that seems
>>>>>>> like it would take less time than uploading the RAW 
>>>>>>> images to S3.
>>>>>>> Thoughts?
>>>> Like I said, I think I wrote patches for this. Qemu-img is
>>>> used to convert the RAW RBD into the snapshot, so QCOW2
>>>> should be no problem as output format. But the code in
>>>> CloudStack seems to assume that the snapshot volume is always
>>>> the same format as the source.
>>>> Wido
>>>> P.S.: I'm on vacation in Norway at the moment, so I'm pretty
>>>> laggy with replying.
>>>>>>> Thank You,
>>>>>>> Logan Barfield Tranquil Hosting
Version: GnuPG v1


View raw message