cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Brody <nickbrody2...@gmail.com>
Subject Re: Help me locate the VM owner of (corrupt) VDI - also 'primary footer invalid: invalid cookie' error
Date Tue, 08 Dec 2015 06:43:14 GMT
Thank you

On Monday, December 7, 2015, France <mailinglists@isg.si> wrote:

> Hi Guys,
>
> after battling with cloudstack (4.3.2+) for the whole day with another
> issue (unable to create snapshot’s for one VM, which I also failed to fix)
> I ran into message on XenServer 6.0.2+Hotfixes SMlog (copy pasted at the
> bottom).
>
> Even thou i kinda understand XenServers relations of VM -> VBD -> VDI ->
> SR -> PBD, i failed to find out to which VM does this corrupted .vhd
> snapshot belong.
> Can you please help me find it? Afterwards, we can find the appropriate
> action in order to remove the error and make coalesce work.
>
> The physical file with corruption is:
>
> /dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba
>
> This is the VDI it supposedly belongs to:
> [root@x3 log]# xe vdi-list uuid=02e8b56b-279c-4d5b-8870-b3c5f2255dc7
> params=all
> uuid ( RO)                    : 02e8b56b-279c-4d5b-8870-b3c5f2255dc7
>               name-label ( RW): base copy
>         name-description ( RW):
>            is-a-snapshot ( RO): false
>              snapshot-of ( RO): <not in database>
>                snapshots ( RO):
>            snapshot-time ( RO): 19700101T00:00:00Z
>       allowed-operations (SRO): generate_config; update; resize; destroy;
> clone; copy; snapshot
>       current-operations (SRO):
>                  sr-uuid ( RO): 3b6e386d-8736-6b6b-7006-f3f4df9bd586
>            sr-name-label ( RO): 1c8dbf77-2b23-3244-91ee-5037cb2a55a8
>                vbd-uuids (SRO):
>          crashdump-uuids (SRO):
>             virtual-size ( RO): 21474836480
>     physical-utilisation ( RO): 12582912
>                 location ( RO): 02e8b56b-279c-4d5b-8870-b3c5f2255dc7
>                     type ( RO): User
>                 sharable ( RO): false
>                read-only ( RO): true
>             storage-lock ( RO): false
>                  managed ( RO): false
>                   parent ( RO): <not in database>
>                  missing ( RO): false
>             other-config (MRW):
>            xenstore-data (MRO):
>                sm-config (MRO): vhd-parent:
> 555b38bf-fe63-41b2-9609-e16f4e26b274; vdi_type: vhd; vhd-blocks:
> eJxjYBgFo2AUjIKRCQAFAAAB
>                  on-boot ( RW): persist
>            allow-caching ( RW): false
>          metadata-latest ( RO): false
>         metadata-of-pool ( RO): <not in database>
>                     tags (SRW):
>
> (It supposedly resides on this SR:
> uuid ( RO)                : 3b6e386d-8736-6b6b-7006-f3f4df9bd586
>           name-label ( RW): 1c8dbf77-2b23-3244-91ee-5037cb2a55a8
>     name-description ( RW): 208 - censored
>                 host ( RO): <shared>
>                 type ( RO): lvmoiscsi
>         content-type ( RO): user
>
> Which is on one of many HA iscsi targets mounted on the XenServer cluster.
> uuid ( RO)                  : f471ef16-4670-2f5e-967e-7e98d4721fe7
>              host-uuid ( RO): f6d699aa-f58f-449e-9a51-6e965562f178
>                sr-uuid ( RO): 3b6e386d-8736-6b6b-7006-f3f4df9bd586
>          device-config (MRO): targetIQN: censoredi:storage.censored;
> target: censored; SCSIid: 1target2lun1
>     currently-attached ( RO): true)
>
> Cleaned exempt from SMlog on cluster master:
>
> [8756] 2015-12-07 21:50:45.118758       ['/usr/sbin/vhd-util', 'check',
> '--debug', '-n',
> '/dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba']
> [8756] 2015-12-07 21:50:45.131975       FAILED: (rc 22) stdout: 'primary
> footer invalid: invalid cookie
> /dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba
> appears invalid; dumping metadata
> VHD Footer Summary:
> -------------------
> Cookie              : conectix
> Features            : (0x00000002) <RESV>
> File format version : Major: 1, Minor: 0
> Data offset         : 512
> Timestamp           : Mon Dec  7 18:19:46 2015
> Creator Application : 'tap'
> Creator version     : Major: 1, Minor: 3
> Creator OS          : Unknown!
> Original disk size  : 20480 MB (21474836480 Bytes)
> Current disk size   : 20480 MB (21474836480 Bytes)
> Geometry            : Cyl: 41610, Hds: 16, Sctrs: 63
>                     : = 20479 MB (21474754560 Bytes)
> Disk type           : Differencing hard disk
> Checksum            : 0xffffef85|0xffffef85 (Good!)
> UUID                : 8f25299d-d7cc-44fe-a42b-95b4e9a31a47
> Saved state         : No
> Hidden              : 1
>
> VHD Header Summary:
> -------------------
> Cookie              : cxsparse
> Data offset (unusd) : 18446744073709
> Table offset        : 1536
> Header version      : 0x00010000
> Max BAT size        : 1048576
> Block size          : 2097152 (2 MB)
> Parent name         :
> VG_XenStorage--3b6e386d--8736--6b6b--7006--f3f4df9bd586-VHD--02e8b56b--279c--4d5b--8870--b3c5f2255dc7
> Parent UUID         : 8cece4c9-8c94-47b9-9f7a-cb6023da9b72
> Parent timestamp    : Mon Dec  7 18:19:46 2015
> Checksum            : 0xffffc6ed|0xffffc6ed (Good!)
>
> VHD Parent Locators:
> --------------------
> locator:            : 0
>        code         : PLAT_CODE_MACX
>        data_space   : 512
>        data_length  : 110
>        data_offset  : 4327424
>        decoded name :
> ./VG_XenStorage--3b6e386d--8736--6b6b--7006--f3f4df9bd586-VHD--02e8b56b--279c--4d5b--8870--b3c5f2255dc7
>
> locator:            : 1
>        code         : PLAT_CODE_W2KU
>        data_space   : 512
>        data_length  : 206
>        data_offset  : 4327936
>        decoded name :
> ./VG_XenStorage--3b6e386d--8736--6b6b--7006--f3f4df9bd586-VHD--02e8b56b--279c--4d5b--8870--b3c5f2255dc7
>
> locator:            : 2
>        code         : PLAT_CODE_W2RU
>        data_space   : 512
>        data_length  : 206
>        data_offset  : 4328448
>        decoded name :
> ./VG_XenStorage--3b6e386d--8736--6b6b--7006--f3f4df9bd586-VHD--02e8b56b--279c--4d5b--8870--b3c5f2255dc7
>
> VHD Batmap Summary:
> -------------------
> Batmap offset       : 4196352
> Batmap size (secs)  : 256
> Batmap version      : 0x00010002
> Checksum            : 0xffffff7f|0xffffff7f (Good!)
>
> ', stderr: ''
> [8756] 2015-12-07 21:50:45.132293       ['/usr/sbin/vhd-util', 'query',
> '--debug', '-s', '-n',
> '/dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba']
> [8756] 2015-12-07 21:50:45.145222       SUCCESS
> [8756] 2015-12-07 21:50:45.145502       lock: tried lock
> /var/lock/sm/3b6e386d-8736-6b6b-7006-f3f4df9bd586/sr, acquired: True
> (exists: True)
> [8756] 2015-12-07 21:50:45.145635       lock: released
> /var/lock/sm/3b6e386d-8736-6b6b-7006-f3f4df9bd586/sr
> [8756] 2015-12-07 21:50:45.145752       ['/usr/sbin/lvchange',
> '/dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba',
> '-p', 'r']
> <8756> 2015-12-07 21:50:45.229864
>  *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*
> <8756> 2015-12-07 21:50:45.229919                ***********************
> <8756> 2015-12-07 21:50:45.229969                *  E X C E P T I O N  *
> <8756> 2015-12-07 21:50:45.230017                ***********************
> <8756> 2015-12-07 21:50:45.230077       coalesce: EXCEPTION
> util.SMException, VHD *1a240d45[VHD](20.000G//136.000M|n) corrupted
> <8756> 2015-12-07 21:50:45.230129         File
> "/opt/xensource/sm/cleanup.py", line 1397, in coalesce
>     self._coalesce(vdi)
>   File "/opt/xensource/sm/cleanup.py", line 1587, in _coalesce
>     vdi._doCoalesce()
>   File "/opt/xensource/sm/cleanup.py", line 1050, in _doCoalesce
>     self.parent.validate()
>   File "/opt/xensource/sm/cleanup.py", line 1043, in validate
>     VDI.validate(self, fast)
>   File "/opt/xensource/sm/cleanup.py", line 633, in validate
>     raise util.SMException("VHD %s corrupted" % self)
>
> <8756> 2015-12-07 21:50:45.230181
>  *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*
> <8756> 2015-12-07 21:50:45.230234       Coalesce failed, skipping
>
>
> Sorry for possible mistakes, but after working for 13 hours, my
> concentration has dropped and thank you for your help.
>
> Regards,
> F.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message