cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wido den Hollander <w...@widodh.nl>
Subject Re: RBD snapshot format
Date Sun, 01 Feb 2015 06:54:43 GMT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On 01/30/2015 04:43 PM, Logan Barfield wrote:
> Hi Wido,
> 
> I didn't see the format information for snapshots stored in the 
> database, so I'm assuming it references the associated volume
> format.
> 

Yes, it is.

> I assume the cleanest way to get around that would be to have a 
> separate format field for snapshots, but that wouldn't really make 
> sense just for this one use case.  Another way would be to just
> have the restore (create template from snapshot, etc.) function
> always assume QCOW2 for RBD snapshots, but that's dirty, and would
> cause problems for existing snapshots.
> 

That would be dirty, since I don't think we can actually know if a
snapshpot was from a RBD volume when it's on SS.

Maybe qemu-img can detect the input format? That way we don't have to
specify it.

> It seems like the best method would be to have the restore
> function look at the actual file to get the format information
> instead of pulling it from the database, though I'm not sure if
> that's even possible.
> 

I think that would be the best option indeed.

> Does anyone else have any thoughts or suggestions?
> 
> Thank You,
> 
> Logan Barfield Tranquil Hosting
> 
> 
> On Thu, Jan 29, 2015 at 4:21 PM, Wido den Hollander
> <wido@widodh.nl> wrote:
> 
> 
> On 01/29/2015 10:05 PM, Logan Barfield wrote:
>>>> Hi Wido,
>>>> 
>>>> The relevant code for this is in:
>>>> 
>>>> I saw that the destination format is set to QCOW2 for 
>>>> "createTemplateFromVolume", but in "backupSnapshot" it's set
>>>> to use the format of the source image 
>>>> "destFile.setFormat(snapshotDisk.getFormat());"
>>>> 
>>>> I believe just changing the "backupSnapshot" code to the use
>>>> QCOW2 "destFile.setFormat(PhysicalDiskFormat.QCOW2);" will
>>>> work fine. I can submit a patch for that, but I wanted to
>>>> make sure there wasn't some other reason for the RAW output
>>>> first.
>>>> 
> 
> No, that won't work. Because the management server will then still 
> think it's in RAW format and store that in the database.
> 
> It has been a long time since I looked at this. I know I did some
> work in this area and made some patches.
> 
> Wido
> 
>>>> 
>>>> Thank You,
>>>> 
>>>> Logan Barfield Tranquil Hosting
>>>> 
>>>> 
>>>> On Thu, Jan 29, 2015 at 3:52 PM, Wido den Hollander 
>>>> <wido@widodh.nl> wrote:
>>>> 
>>>> 
>>>> On 01/29/2015 05:16 PM, Logan Barfield wrote:
>>>>>>> I was doing some testing earlier this week on a KVM
>>>>>>> cluster, and noticed that when using S3 for secondary
>>>>>>> storage snapshots on RBD primary storage take a lot
>>>>>>> longer than they do with NFS primary storage.
>>>>>>> 
>>>>>>> I think this is due to the NFS snapshots being 
>>>>>>> output/uploaded in QCOW2 format, while RBD snapshots
>>>>>>> are output in RAW format.  It appears that even when
>>>>>>> using sparse RAW files they are uploaded to S3 with the
>>>>>>> 'allocated' size instead of the sparse size.
>>>>>>> 
>>>> 
>>>> The problem here lies in the code in CloudStack. Since the
>>>> source volume is in RAW format, the snapshot also becomes
>>>> QCOW2.
>>>> 
>>>> I remember that I wrote some patches for this, you might want
>>>> to look in the Git history.
>>>> 
>>>>>>> My question is: Is there a good reason for RBD
>>>>>>> snapshots to be output as RAW instead of QCOW2?  It
>>>>>>> appears that QCOW2 templates are automatically
>>>>>>> converted when deploying them as RBD, so that shouldn't
>>>>>>> be a problem.  The only downside I can think of would
>>>>>>> be having to convert the RAW RBD data to QCOW2 as an
>>>>>>> extra step when creating snapshots, but even that seems
>>>>>>> like it would take less time than uploading the RAW 
>>>>>>> images to S3.
>>>>>>> 
>>>>>>> Thoughts?
>>>>>>> 
>>>> 
>>>> Like I said, I think I wrote patches for this. Qemu-img is
>>>> used to convert the RAW RBD into the snapshot, so QCOW2
>>>> should be no problem as output format. But the code in
>>>> CloudStack seems to assume that the snapshot volume is always
>>>> the same format as the source.
>>>> 
>>>> Wido
>>>> 
>>>> P.S.: I'm on vacation in Norway at the moment, so I'm pretty
>>>> laggy with replying.
>>>> 
>>>>>>> 
>>>>>>> Thank You,
>>>>>>> 
>>>>>>> Logan Barfield Tranquil Hosting
>>>>>>> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJUzc2zAAoJEAGbWC3bPspCLFAP/0FuGqOTlo4DW40tP4v70zQ5
cjF1s9rg9Afd4OGc6KmcG2OxR/A68xMuHxJCPwVZjFlcNLMEWx8goxmJd9qLQaMM
es1xHynmSO3owhA9d7bRhvfL4M9mf/w/oSn1IQZ5l9v7bJX9y5iGuPkMPlBT5yej
yFWA92AAEVgne1J/E4JvoZbDoyAYVecrDPmFD8oP/9R2I7MlUr5134aShBIhJPM/
UgsY06dKfAWI2cW5vZxzw4haNTk68BMgjp6IBkirHqLL1wZa7R83c8/ygXsbvStJ
z5bLrxtLtwy6d3utjwJkfdtdnAfuiqZFOKdgxvhlS30Q0Y2XGLXfA3+DLVGmDBF5
xksxiAvhpvxkJIMYZx5icFJmNVu5NiaNn6yKRTkJzA687emF4fSjn1pgTHMp+hhB
gkZcYiazrrdFU0ibFkQMNWhT80Pc9spcJEMFkX32GnXTK8EOR7hqRpGrS1LLQjnk
XrukOOOjlHnQnRS8kTP9y2Rr5zBUnB6kVwd8DYCJGYdQUPlHaMOGPC9che7HzmFg
qzNPHm/WakWotlAldYOPFzmoV16jzF+j6lpgnREZMTq3DNVk6fRH4vzjIvlWzsbF
Rrq+7KbNxYc0SJfmTBh0jV/Zpm85ed0kYzXBnekq8ryItF1e2P5t4zoVcC7XHb9n
Y2Qvi+YYYZPW6fYIiznr
=TcvA
-----END PGP SIGNATURE-----

Mime
View raw message