cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From France <mailingli...@isg.si>
Subject Help me locate the VM owner of (corrupt) VDI - also 'primary footer invalid: invalid cookie' error
Date Mon, 07 Dec 2015 22:14:35 GMT
Hi Guys,

after battling with cloudstack (4.3.2+) for the whole day with another issue (unable to create
snapshot’s for one VM, which I also failed to fix) I ran into message on XenServer 6.0.2+Hotfixes
SMlog (copy pasted at the bottom).

Even thou i kinda understand XenServers relations of VM -> VBD -> VDI -> SR ->
PBD, i failed to find out to which VM does this corrupted .vhd snapshot belong.
Can you please help me find it? Afterwards, we can find the appropriate action in order to
remove the error and make coalesce work.

The physical file with corruption is:
/dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba

This is the VDI it supposedly belongs to:
[root@x3 log]# xe vdi-list uuid=02e8b56b-279c-4d5b-8870-b3c5f2255dc7 params=all
uuid ( RO)                    : 02e8b56b-279c-4d5b-8870-b3c5f2255dc7
              name-label ( RW): base copy
        name-description ( RW): 
           is-a-snapshot ( RO): false
             snapshot-of ( RO): <not in database>
               snapshots ( RO): 
           snapshot-time ( RO): 19700101T00:00:00Z
      allowed-operations (SRO): generate_config; update; resize; destroy; clone; copy; snapshot
      current-operations (SRO): 
                 sr-uuid ( RO): 3b6e386d-8736-6b6b-7006-f3f4df9bd586
           sr-name-label ( RO): 1c8dbf77-2b23-3244-91ee-5037cb2a55a8
               vbd-uuids (SRO): 
         crashdump-uuids (SRO): 
            virtual-size ( RO): 21474836480
    physical-utilisation ( RO): 12582912
                location ( RO): 02e8b56b-279c-4d5b-8870-b3c5f2255dc7
                    type ( RO): User
                sharable ( RO): false
               read-only ( RO): true
            storage-lock ( RO): false
                 managed ( RO): false
                  parent ( RO): <not in database>
                 missing ( RO): false
            other-config (MRW): 
           xenstore-data (MRO): 
               sm-config (MRO): vhd-parent: 555b38bf-fe63-41b2-9609-e16f4e26b274; vdi_type:
vhd; vhd-blocks: eJxjYBgFo2AUjIKRCQAFAAAB
                 on-boot ( RW): persist
           allow-caching ( RW): false
         metadata-latest ( RO): false
        metadata-of-pool ( RO): <not in database>
                    tags (SRW): 

(It supposedly resides on this SR:
uuid ( RO)                : 3b6e386d-8736-6b6b-7006-f3f4df9bd586
          name-label ( RW): 1c8dbf77-2b23-3244-91ee-5037cb2a55a8
    name-description ( RW): 208 - censored
                host ( RO): <shared>
                type ( RO): lvmoiscsi
        content-type ( RO): user

Which is on one of many HA iscsi targets mounted on the XenServer cluster.
uuid ( RO)                  : f471ef16-4670-2f5e-967e-7e98d4721fe7
             host-uuid ( RO): f6d699aa-f58f-449e-9a51-6e965562f178
               sr-uuid ( RO): 3b6e386d-8736-6b6b-7006-f3f4df9bd586
         device-config (MRO): targetIQN: censoredi:storage.censored; target: censored; SCSIid:
1target2lun1
    currently-attached ( RO): true)

Cleaned exempt from SMlog on cluster master:

[8756] 2015-12-07 21:50:45.118758	['/usr/sbin/vhd-util', 'check', '--debug', '-n', '/dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba']
[8756] 2015-12-07 21:50:45.131975	FAILED: (rc 22) stdout: 'primary footer invalid: invalid
cookie
/dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba
appears invalid; dumping metadata
VHD Footer Summary:
-------------------
Cookie              : conectix
Features            : (0x00000002) <RESV>
File format version : Major: 1, Minor: 0
Data offset         : 512
Timestamp           : Mon Dec  7 18:19:46 2015
Creator Application : 'tap'
Creator version     : Major: 1, Minor: 3
Creator OS          : Unknown!
Original disk size  : 20480 MB (21474836480 Bytes)
Current disk size   : 20480 MB (21474836480 Bytes)
Geometry            : Cyl: 41610, Hds: 16, Sctrs: 63
                    : = 20479 MB (21474754560 Bytes)
Disk type           : Differencing hard disk
Checksum            : 0xffffef85|0xffffef85 (Good!)
UUID                : 8f25299d-d7cc-44fe-a42b-95b4e9a31a47
Saved state         : No
Hidden              : 1

VHD Header Summary:
-------------------
Cookie              : cxsparse
Data offset (unusd) : 18446744073709
Table offset        : 1536
Header version      : 0x00010000
Max BAT size        : 1048576
Block size          : 2097152 (2 MB)
Parent name         : VG_XenStorage--3b6e386d--8736--6b6b--7006--f3f4df9bd586-VHD--02e8b56b--279c--4d5b--8870--b3c5f2255dc7
Parent UUID         : 8cece4c9-8c94-47b9-9f7a-cb6023da9b72
Parent timestamp    : Mon Dec  7 18:19:46 2015
Checksum            : 0xffffc6ed|0xffffc6ed (Good!)

VHD Parent Locators:
--------------------
locator:            : 0
       code         : PLAT_CODE_MACX
       data_space   : 512
       data_length  : 110
       data_offset  : 4327424
       decoded name : ./VG_XenStorage--3b6e386d--8736--6b6b--7006--f3f4df9bd586-VHD--02e8b56b--279c--4d5b--8870--b3c5f2255dc7

locator:            : 1
       code         : PLAT_CODE_W2KU
       data_space   : 512
       data_length  : 206
       data_offset  : 4327936
       decoded name : ./VG_XenStorage--3b6e386d--8736--6b6b--7006--f3f4df9bd586-VHD--02e8b56b--279c--4d5b--8870--b3c5f2255dc7

locator:            : 2
       code         : PLAT_CODE_W2RU
       data_space   : 512
       data_length  : 206
       data_offset  : 4328448
       decoded name : ./VG_XenStorage--3b6e386d--8736--6b6b--7006--f3f4df9bd586-VHD--02e8b56b--279c--4d5b--8870--b3c5f2255dc7

VHD Batmap Summary:
-------------------
Batmap offset       : 4196352
Batmap size (secs)  : 256
Batmap version      : 0x00010002
Checksum            : 0xffffff7f|0xffffff7f (Good!)

', stderr: ''
[8756] 2015-12-07 21:50:45.132293	['/usr/sbin/vhd-util', 'query', '--debug', '-s', '-n', '/dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba']
[8756] 2015-12-07 21:50:45.145222	SUCCESS
[8756] 2015-12-07 21:50:45.145502	lock: tried lock /var/lock/sm/3b6e386d-8736-6b6b-7006-f3f4df9bd586/sr,
acquired: True (exists: True)
[8756] 2015-12-07 21:50:45.145635	lock: released /var/lock/sm/3b6e386d-8736-6b6b-7006-f3f4df9bd586/sr
[8756] 2015-12-07 21:50:45.145752	['/usr/sbin/lvchange', '/dev/VG_XenStorage-3b6e386d-8736-6b6b-7006-f3f4df9bd586/VHD-1a240d45-ee0a-4c30-809b-3114dfaf85ba',
'-p', 'r']
<8756> 2015-12-07 21:50:45.229864	*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*
<8756> 2015-12-07 21:50:45.229919	         ***********************
<8756> 2015-12-07 21:50:45.229969	         *  E X C E P T I O N  *
<8756> 2015-12-07 21:50:45.230017	         ***********************
<8756> 2015-12-07 21:50:45.230077	coalesce: EXCEPTION util.SMException, VHD *1a240d45[VHD](20.000G//136.000M|n)
corrupted
<8756> 2015-12-07 21:50:45.230129	  File "/opt/xensource/sm/cleanup.py", line 1397,
in coalesce
    self._coalesce(vdi)
  File "/opt/xensource/sm/cleanup.py", line 1587, in _coalesce
    vdi._doCoalesce()
  File "/opt/xensource/sm/cleanup.py", line 1050, in _doCoalesce
    self.parent.validate()
  File "/opt/xensource/sm/cleanup.py", line 1043, in validate
    VDI.validate(self, fast)
  File "/opt/xensource/sm/cleanup.py", line 633, in validate
    raise util.SMException("VHD %s corrupted" % self)

<8756> 2015-12-07 21:50:45.230181	*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*
<8756> 2015-12-07 21:50:45.230234	Coalesce failed, skipping


Sorry for possible mistakes, but after working for 13 hours, my concentration has dropped
and thank you for your help.

Regards,
F.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message