cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rafael Weingärtner <rafaelweingart...@gmail.com>
Subject Re: Strange size of template from snapshot on XenServer
Date Wed, 01 Nov 2017 19:16:44 GMT
Well, I could not reproduce the problem in ACS 4.9.2.0. So, before fixing
anything, the person that might take on this should first find the root of
the problem.

I understand what you are saying that it is a critical problem, but you
have to understand that this is an open-source and free software.

Having said that, ALL of my work here has been pro-bono; even though I am a
committer and PMC, I am not willing to dive deeper into this issue now as
it does not affect me. It does not mean that I would not help people if I
have time, as you can notice whenever I have some free time I try to be
around and guide or check things for other people in this list.

If this is a huge problem for you right now, I suggest you looking for
companies that provide enterprise support for ACS.

On Wed, Nov 1, 2017 at 7:35 PM, Lotic Lists <lists@lotic.com.br> wrote:

> Hi Rafael
>
> Tested on ACS 4.5.2.2, 4.9.2.0, 4.9.3.0 and 4.10. All versions with same
> issue
>
> Description and steps to reproduce
> https://issues.apache.org/jira/browse/CLOUDSTACK-10128
>
> Again, it is a critical problem for production environments.
>
> Regards
> Marcelo
>
> -----Original Message-----
> From: Rafael Weingärtner [mailto:rafaelweingartner@gmail.com]
> Sent: terça-feira, 31 de outubro de 2017 17:10
> To: dev@cloudstack.apache.org
> Subject: Re: Strange size of template from snapshot on XenServer
>
> Yes, it is a big problem. I have not checked the code that creates the
> template, but I am not sure if the problem you are experiencing is related
> to 9025. It might be a problem in some other place.
>
> My suggestion is for you to create a Jira ticket detailing the situation
> and wait until someone fixes it. Have you tested on ACS 4.10? If 4.9.2.0
> works, it might be something that went only into 4.9.3.0.
>
> On Tue, Oct 31, 2017 at 8:04 PM, Lotic Lists <lists@lotic.com.br> wrote:
>
> > I tested on a fresh environment, acs 4.9.3.0, xenserver 6.5 with all
> > patches and primary storage NFS, we have same problem
> >
> > - Snapshot files
> > [root@acs01 203]# ls -lh /exports/secondary/snapshots/2/6/ total 2.8G
> > -rw-r--r-- 1 root root 1.7G Oct 31 16:48
> > cad96f0c-de49-4255-b0a0-9aea5e2297cb.vhd
> > <-- snap-base
> > -rw-r--r-- 1 root root 1.1G Oct 31 16:49
> > fa4e7e4d-531d-42dd-8a80-df6b274584a8.vhd
> > <-- snap-diff
> > https://i.imgur.com/bWupsAS.png
> >
> > [root@acs01 203]# md5sum /exports/secondary/snapshots/
> > 2/6/fa4e7e4d-531d-42dd-8a80-df6b274584a8.vhd
> > 17eec8d4f6d4c128b34b9e2e1876ceb7  /exports/secondary/snapshots/
> > 2/6/fa4e7e4d-531d-42dd-8a80-df6b274584a8.vhd
> >
> > - Template files
> > [root@acs01 203]# ls -lh /exports/secondary/template/tmpl/2/203
> > total 2.0G
> > -rw-r--r-- 1 root root 1.1G Oct 31 16:50 e9def45d-1b9e-4c2e-b843-
> > 852fe40f00b2.vhd
> > -rw-r--r-- 1 root root  303 Oct 31 16:50 template.properties
> >
> > [root@acs01 203]# md5sum /exports/secondary/template/
> > tmpl/2/203/e9def45d-1b9e-4c2e-b843-852fe40f00b2.vhd
> > 17eec8d4f6d4c128b34b9e2e1876ceb7  /exports/secondary/template/
> > tmpl/2/203/e9def45d-1b9e-4c2e-b843-852fe40f00b2.vhd
> >
> > It's a huge problem!!! I think issue 9025 is not resolved
> >
> > Regards
> > Marcelo
> >
> > -----Original Message-----
> > From: Rafael Weingärtner [mailto:rafaelweingartner@gmail.com]
> > Sent: terça-feira, 31 de outubro de 2017 16:00
> > To: users@cloudstack.apache.org
> > Subject: Re: Strange size of template from snapshot on XenServer
> >
> > Well, reading the description of the issue it seems to be related to
> > the problem you are describing. However, I compared the classes that
> > were changed with PR (https://github.com/apache/cloudstack/pull/1176)
> > and they are the same in ACS 4.9.2.0 and 4.9.3.0.
> >
> > On Tue, Oct 31, 2017 at 6:36 PM, Lotic Lists <lists@lotic.com.br> wrote:
> >
> > > Hi Rafael, thanks for test.
> > >
> > > I found the issue
> > > https://issues.apache.org/jira/browse/CLOUDSTACK-9025
> > > I think is a same case. I will up a new lab environment now for
> > > simulate the issue
> > >
> > > Regards.
> > > Marcelo
> > >
> > > -----Original Message-----
> > > From: Rafael Weingärtner [mailto:rafaelweingartner@gmail.com]
> > > Sent: segunda-feira, 30 de outubro de 2017 12:08
> > > To: users@cloudstack.apache.org
> > > Subject: Re: Strange size of template from snapshot on XenServer
> > >
> > > I just did, and the size of the template is the size of the root
> > > disk
> > > + the 1GB file I created in the test vm.
> > >
> > > The system I tested this is an ACS 4.9.2, XenServer 6.5, Primary Stg
> > > ISCSI
> > >
> > > On Mon, Oct 30, 2017 at 10:20 AM, Lotic Lists <lists@lotic.com.br>
> > wrote:
> > >
> > > > Good morning
> > > >
> > > > Guys, who could execute the test below?
> > > >
> > > > 1. Create a manual snapshot of a volume 2. Create a file (1GB)
> > > > with dd on VM 3. Create second snapshot of a volume 4. Convert
> > > > latest snapshot to a template 5. Verify the size of vhd of new
> > > > template in secondary storage.
> > > >
> > > > Here the template have 1GB, exactly the size of latest snapshot.
> > > >
> > > > ACS 4.9.3, XenServer 6.5, Primary Stg ISCSI
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Lotic Lists [mailto:lists@lotic.com.br]
> > > > Sent: quinta-feira, 26 de outubro de 2017 15:03
> > > > To: users@cloudstack.apache.org
> > > > Subject: RE: Strange size of template from snapshot on XenServer
> > > >
> > > > I test the same case on other environment, create template from
> > > > snapshot, acs 4.9.2.0 and XenServer 6.5/iscsi
> > > >
> > > > 1. Was created two snapshots
> > > >
> > > > # ls -lh /var/cloud_mount/e5921d9a-70b0-3501-9fda-6097e721fffb/
> > > > snapshots/131/617/
> > > > total 3.0G
> > > > -rw-r--r-- 1 4294967294 root  13G Oct 26 13:46
> > > > 42c7c51d-4aa7-40d2-b5eb- cb56fc963974.vhd
> > > > -rw-r--r-- 1 4294967294 root 1.2G Oct 26 14:42
> > > > 53b33a5a-a7d9-4efa-b126- 38eec5482b05.vhd
> > > >
> > > > https://i.imgur.com/cSHZAJ3.png
> > > >
> > > > 2. create a template from latest snapshot
> > > >
> > > > # ls -lh
> > > > total 37M
> > > > -rw-r--r-- 1 4294967294 root 1.2G Oct 26 14:43
> > > > 637604a6-3085-444e-ba09- b9aab11a2c16.vhd
> > > > -rw-r--r-- 1 4294967294 root  303 Oct 26 14:43 template.properties
> > > >
> > > > # cat template.properties
> > > > #Thu Oct 26 16:43:54 UTC 2017
> > > > filename=637604a6-3085-444e-ba09-b9aab11a2c16.vhd
> > > > id=1
> > > > vhd=true
> > > > vhd.filename=637604a6-3085-444e-ba09-b9aab11a2c16.vhd
> > > > public=true
> > > > uniquename=637604a6-3085-444e-ba09-b9aab11a2c16
> > > > vhd.virtualsize=21474836480
> > > > virtualsize=21474836480
> > > > hvm=
> > > > vhd.size=1197752832
> > > > size=1197752832
> > > >
> > > > # md5sum /var/cloud_mount/e5921d9a-70b0-3501-9fda-6097e721fffb/
> > > > snapshots/131/617/53b33a5a-a7d9-4efa-b126-38eec5482b05.vhd
> > > > 9d2078a6f41ca5e1eab824848c06df53  /var/cloud_mount/e5921d9a-
> > > > 70b0-3501-9fda-6097e721fffb/snapshots/131/617/53b33a5a-
> > > > a7d9-4efa-b126-38eec5482b05.vhd
> > > >
> > > > # md5sum /var/cloud_mount/e5921d9a-70b0-3501-9fda-6097e721fffb/
> > > > template/tmpl/2/294/637604a6-3085-444e-ba09-b9aab11a2c16.vhd
> > > > 9d2078a6f41ca5e1eab824848c06df53  /var/cloud_mount/e5921d9a-
> > > > 70b0-3501-9fda-6097e721fffb/template/tmpl/2/294/637604a6-
> > > > 3085-444e-ba09-b9aab11a2c16.vhd
> > > >
> > > > ACS not merge files of snapshots in a single file for template, I
> > > > think it's a bug.
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Rafael Weingärtner [mailto:rafaelweingartner@gmail.com]
> > > > Sent: quarta-feira, 25 de outubro de 2017 22:51
> > > > To: users@cloudstack.apache.org
> > > > Subject: Re: Strange size of template from snapshot on XenServer
> > > >
> > > > It is easy to test, get one of these files and try to use them in
> > > > Virtualbox. If they run, they do not require anything else.
> > > >
> > > > On Wed, Oct 25, 2017 at 10:49 PM, Lotic Lists <lists@lotic.com.br>
> > > wrote:
> > > >
> > > > > Sure but files on secondary storage don't have parents, I don't
> > > > > know if ACS can connect standalone files.
> > > > >
> > > > > vhd-util scan -f -m'*.vhd' -p
> > > > > vhd=14818597-55bb-49be-9ace-761e9e01c074.vhd
> > > > > capacity=311385128960
> > > > > size=304124744192 hidden=0 parent=none
> > > > > vhd=bbd9e9b5-53fb-44a2-afbe-f262396a2d84.vhd
> > > > > capacity=311385128960
> > > > > size=17770869248 hidden=0 parent=none
> > > > > vhd=3b90bbb2-7ce5-41f6-9f7e-fd0ea061cb2d.vhd
> > > > > capacity=311385128960
> > > > > size=52199817728 hidden=0 parent=none
> > > > > vhd=4b450217-c9ad-45a2-946b-de3cb323469b.vhd
> > > > > capacity=311385128960
> > > > > size=51615670784 hidden=0 parent=none
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Rafael Weingärtner [mailto:rafaelweingartner@gmail.com]
> > > > > Sent: quarta-feira, 25 de outubro de 2017 22:35
> > > > > To: users@cloudstack.apache.org
> > > > > Subject: Re: Strange size of template from snapshot on XenServer
> > > > >
> > > > > If it was not executing coalesce you would see the parent
> > > > > reference when you listed the hierarchy, right?
> > > > >
> > > > > On Wed, Oct 25, 2017 at 10:25 PM, Lotic Lists
> > > > > <lists@lotic.com.br>
> > > > wrote:
> > > > >
> > > > > > I think ACS preserve all VHDs for coalesce but coalesce not
> > > > > > occur when create a template or before starting next snapshot
> > scheduled.
> > > > > > If I create a template from any snapshots (2) showing in GUI,
> > > > > > the template have 49GB. 49 GB is the size of the 2 latest
> > > > > > files on secondary
> > > > > storage.
> > > > > >
> > > > > > What do you think? Bug?
> > > > > >
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Rafael Weingärtner [mailto:rafaelweingartner@gmail.com]
> > > > > > Sent: quarta-feira, 25 de outubro de 2017 22:07
> > > > > > To: users@cloudstack.apache.org
> > > > > > Subject: Re: Strange size of template from snapshot on
> > > > > > XenServer
> > > > > >
> > > > > > Aha, The two destroyed entries are showing " removed: NULL".
> > > > > >
> > > > > > In ACS one thing is the "destroying" of a resource, another
is
> > > > > > the removal of this resource from the system. I believe there
> > > > > > is something to do with the expunge interval, but I am not
> > > > > > sure if the snapshots complete removal also happens within the
> > > > > > expunge interval of user VMs or if it is configured by something
> else.
> > > > > >
> > > > > >
> > > > > > On Wed, Oct 25, 2017 at 10:03 PM, Lotic Lists
> > > > > > <lists@lotic.com.br>
> > > > > wrote:
> > > > > >
> > > > > > > I not start manual snapshot, just scheduled with keep=2
> > > > > > >
> > > > > > > GUI showing 2 snapshots, database showing 4 entries, 2
> > > > > > > Destroyed and
> > > > > > > 2 BackedUp
> > > > > > >
> > > > > > > *************************** 132. row
> ***************************
> > > > > > >               id: 6380
> > > > > > >   data_center_id: 2
> > > > > > >       account_id: 64
> > > > > > >        domain_id: 49
> > > > > > >        volume_id: 2102
> > > > > > > disk_offering_id: 230
> > > > > > >           status: Destroyed
> > > > > > >             path: NULL
> > > > > > >             name: FullCRM_ROOT-542_20171022030205
> > > > > > >             uuid: 1294cdb7-01bf-4e76-b4c0-11b3c6b395e9
> > > > > > >    snapshot_type: 4
> > > > > > > type_description: DAILY
> > > > > > >             size: 311385128960
> > > > > > >          created: 2017-10-22 03:02:05
> > > > > > >          removed: NULL
> > > > > > >   backup_snap_id: NULL
> > > > > > >         swift_id: NULL
> > > > > > >       sechost_id: NULL
> > > > > > >     prev_snap_id: NULL
> > > > > > >  hypervisor_type: XenServer
> > > > > > >          version: 2.2
> > > > > > >            s3_id: NULL
> > > > > > >         min_iops: NULL
> > > > > > >         max_iops: NULL
> > > > > > > *************************** 133. row
> ***************************
> > > > > > >               id: 6416
> > > > > > >   data_center_id: 2
> > > > > > >       account_id: 64
> > > > > > >        domain_id: 49
> > > > > > >        volume_id: 2102
> > > > > > > disk_offering_id: 230
> > > > > > >           status: Destroyed
> > > > > > >             path: NULL
> > > > > > >             name: FullCRM_ROOT-542_20171023030205
> > > > > > >             uuid: 0a8067a3-ad15-40b6-8b16-c1881d7d9f41
> > > > > > >    snapshot_type: 4
> > > > > > > type_description: DAILY
> > > > > > >             size: 311385128960
> > > > > > >          created: 2017-10-23 03:02:05
> > > > > > >          removed: NULL
> > > > > > >   backup_snap_id: NULL
> > > > > > >         swift_id: NULL
> > > > > > >       sechost_id: NULL
> > > > > > >     prev_snap_id: NULL
> > > > > > >  hypervisor_type: XenServer
> > > > > > >          version: 2.2
> > > > > > >            s3_id: NULL
> > > > > > >         min_iops: NULL
> > > > > > >         max_iops: NULL
> > > > > > > *************************** 134. row
> ***************************
> > > > > > >               id: 6455
> > > > > > >   data_center_id: 2
> > > > > > >       account_id: 64
> > > > > > >        domain_id: 49
> > > > > > >        volume_id: 2102
> > > > > > > disk_offering_id: 230
> > > > > > >           status: BackedUp
> > > > > > >             path: NULL
> > > > > > >             name: FullCRM_ROOT-542_20171024030207
> > > > > > >             uuid: 1ca646e5-a155-4a59-aa31-5889eec7536f
> > > > > > >    snapshot_type: 4
> > > > > > > type_description: DAILY
> > > > > > >             size: 311385128960
> > > > > > >          created: 2017-10-24 03:02:07
> > > > > > >          removed: NULL
> > > > > > >   backup_snap_id: NULL
> > > > > > >         swift_id: NULL
> > > > > > >       sechost_id: NULL
> > > > > > >     prev_snap_id: NULL
> > > > > > >  hypervisor_type: XenServer
> > > > > > >          version: 2.2
> > > > > > >            s3_id: NULL
> > > > > > >         min_iops: NULL
> > > > > > >         max_iops: NULL
> > > > > > > *************************** 135. row
> ***************************
> > > > > > >               id: 6497
> > > > > > >   data_center_id: 2
> > > > > > >       account_id: 64
> > > > > > >        domain_id: 49
> > > > > > >        volume_id: 2102
> > > > > > > disk_offering_id: 230
> > > > > > >           status: BackedUp
> > > > > > >             path: NULL
> > > > > > >             name: FullCRM_ROOT-542_20171025030208
> > > > > > >             uuid: 759d61dc-e3db-4815-b68b-b90904ccbda5
> > > > > > >    snapshot_type: 4
> > > > > > > type_description: DAILY
> > > > > > >             size: 311385128960
> > > > > > >          created: 2017-10-25 03:02:08
> > > > > > >          removed: NULL
> > > > > > >   backup_snap_id: NULL
> > > > > > >         swift_id: NULL
> > > > > > >       sechost_id: NULL
> > > > > > >     prev_snap_id: NULL
> > > > > > >  hypervisor_type: XenServer
> > > > > > >          version: 2.2
> > > > > > >            s3_id: NULL
> > > > > > >         min_iops: NULL
> > > > > > >         max_iops: NULL
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Rafael Weingärtner
> > > > > > > [mailto:rafaelweingartner@gmail.com]
> > > > > > > Sent: quarta-feira, 25 de outubro de 2017 21:50
> > > > > > > To: users@cloudstack.apache.org
> > > > > > > Subject: Re: Strange size of template from snapshot on
> > > > > > > XenServer
> > > > > > >
> > > > > > > The file "14818597-55bb-49be-9ace-761e9e01c074.vhd" seems
> > > > > > > more like a manual snapshot. It was created 2 hours after
the
> "normal"
> > > > > > > time. I would check these snapshots in ACS and see how
they
> > > > > > > are
> > > > presented.
> > > > > > > BTW, how many snapshots is ACS showing for this volume?
I
> > > > > > > would also check the "snapshots" table for the volume you
> > > > > > > have configured
> > > > them.
> > > > > > > select * from snapshots where volume_id = ?
> > > > > > >
> > > > > > > you may also filter for removed:
> > > > > > > select * from snapshots where volume_id = ? and removed
is
> > > > > > > null
> > > > > > >
> > > > > > > On Wed, Oct 25, 2017 at 9:39 PM, Lotic Lists
> > > > > > > <lists@lotic.com.br>
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi Rafael, I check
> > > > > > > >
> > > > > > > > # ls -lrt
> > > > > > > > total 285368382
> > > > > > > > -rw-r--r-- 1 root root 304124744192 Oct 22 03:28
> > > > > > > > 14818597-55bb-49be-9ace- 761e9e01c074.vhd
> > > > > > > > -rw-r--r-- 1 root root  17770869248 Oct 23 01:10
> > > > > > > > bbd9e9b5-53fb-44a2-afbe- f262396a2d84.vhd
> > > > > > > > -rw-r--r-- 1 root root  52199817728 Oct 24 01:30
> > > > > > > > 3b90bbb2-7ce5-41f6-9f7e- fd0ea061cb2d.vhd
> > > > > > > > -rw-r--r-- 1 root root  51615670784 Oct 25 01:28
> > > > > > > > 4b450217-c9ad-45a2-946b- de3cb323469b.vhd
> > > > > > > >
> > > > > > > > # vhd-util scan -f -m'*.vhd' -p
> > > > > > > > vhd=14818597-55bb-49be-9ace-761e9e01c074.vhd
> > > > > > > > capacity=311385128960
> > > > > > > > size=304124744192 hidden=0 parent=none
> > > > > > > > vhd=bbd9e9b5-53fb-44a2-afbe-f262396a2d84.vhd
> > > > > > > > capacity=311385128960
> > > > > > > > size=17770869248 hidden=0 parent=none
> > > > > > > > vhd=3b90bbb2-7ce5-41f6-9f7e-fd0ea061cb2d.vhd
> > > > > > > > capacity=311385128960
> > > > > > > > size=52199817728 hidden=0 parent=none
> > > > > > > > vhd=4b450217-c9ad-45a2-946b-de3cb323469b.vhd
> > > > > > > > capacity=311385128960
> > > > > > > > size=51615670784 hidden=0 parent=none
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Rafael Weingärtner
> > > > > > > > [mailto:rafaelweingartner@gmail.com]
> > > > > > > > Sent: quarta-feira, 25 de outubro de 2017 20:44
> > > > > > > > To: users@cloudstack.apache.org
> > > > > > > > Subject: Re: Strange size of template from snapshot
on
> > > > > > > > XenServer
> > > > > > > >
> > > > > > > > Did you check the hierarchy of those snapshot files
in
> > > > > > > > your secondary storage?
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Oct 25, 2017 at 7:44 PM, Lotic Lists
> > > > > > > > <lists@lotic.com.br>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi all
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Anyone identify problems with templates from
snapshots
> > > > > > > > > on
> > > > > XenServer?
> > > > > > > > >
> > > > > > > > > I created a recurrent snapshot, first VHD on
secondary
> > > > > > > > > storage have a similar size of volume. If I create
a
> > > > > > > > > template from latest snapshot, the size of VHD
of
> > > > > > > > > template is smaller equal from last snapshot,
I think
> > > > > > > > > CloudStack/Xenserver not coalesce VHD files of
secondary
> > > > > > > > > storage for create the template
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I keep two recurrent snapshots, on secondary
storage
> > > > > > > > > have four
> > > > > files.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Environment is ACS 4.9.3.0 and XenServer 6.5,
primary
> > > > > > > > > storage is PreSetup with ISCSI.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thanks
> > > > > > > > >
> > > > > > > > > Marcelo
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Rafael Weingärtner
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Rafael Weingärtner
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Rafael Weingärtner
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Rafael Weingärtner
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Rafael Weingärtner
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Rafael Weingärtner
> > >
> > >
> >
> >
> > --
> > Rafael Weingärtner
> >
> >
>
>
> --
> Rafael Weingärtner
>
>


-- 
Rafael Weingärtner

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message