cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yiping Zhang <yzh...@marketo.com>
Subject [Urgent]: corrupt DB after VM live migration with storage migration
Date Wed, 04 May 2016 22:22:32 GMT
Hi, all:

I am in a situation that I need some help:

I did a live migration with storage migration required for a production VM instance from one
cluster to another.  The first migration attempt failed after some time, but the second attempt
succeeded. During all this time the VM instance is accessible (and it is still up and running).
 However, when I use my api script to query volumes, it still reports that the volume is on
the old cluster’s primary storage.  If I shut down this VM,  I am afraid that it won’t
start again as it would try to use non-existing volumes.

Checking database, sure enough, the DB still has old info about these volumes:


mysql> select id,name from storage_pool where id=1 or id=8;

+----+------------------+

| id | name             |

+----+------------------+

|  1 | abprod-primary1  |

|  8 | abprod-p1c2-pri1 |

+----+------------------+

2 rows in set (0.01 sec)


Here the old cluster’s primary storage has id=1, and the new cluster’s primary storage
has id=8.


Here are the entries with wrong info in volumes table:


mysql> select id,name, uuid, path,pool_id, removed from volumes where name='ROOT-97' or
name='DATA-97';

+-----+---------+--------------------------------------+--------------------------------------+---------+---------------------+

| id  | name    | uuid                                 | path                            
    | pool_id | removed             |

+-----+---------+--------------------------------------+--------------------------------------+---------+---------------------+

| 124 | ROOT-97 | 224bf673-fda8-4ccc-9c30-fd1068aee005 | 5d1ab4ef-2629-4384-a56a-e2dc1055d032
|       1 | NULL                |

| 125 | DATA-97 | d385d635-9230-4130-8d1f-702dbcf0f22c | 6b75496d-5907-46c3-8836-5618f11dac8e
|       1 | NULL                |

| 316 | ROOT-97 | 691b5c12-7ec4-408d-b66f-1ff041f149c1 | NULL                            
    |       8 | 2016-05-03 06:10:40 |

| 317 | ROOT-97 | 8ba29fcf-a81a-4ca0-9540-0287230f10c7 | NULL                            
    |       8 | 2016-05-03 06:10:45 |

+-----+---------+--------------------------------------+--------------------------------------+---------+---------------------+

4 rows in set (0.01 sec)

On the xenserver of old cluster, the volumes do not exist:


[root@abmpc-hv01 ~]# xe vdi-list name-label='ROOT-97'

[root@abmpc-hv01 ~]# xe vdi-list name-label='DATA-97'

[root@abmpc-hv01 ~]#

But the volumes are on the new cluster’s primary storage:


[root@abmpc-hv04 ~]# xe vdi-list name-label=ROOT-97

uuid ( RO)                : a253b217-8cdc-4d4a-a111-e5b6ad48a1d5

          name-label ( RW): ROOT-97

    name-description ( RW):

             sr-uuid ( RO): 6d4bea51-f253-3b43-2f2f-6d7ba3261ed3

        virtual-size ( RO): 34359738368

            sharable ( RO): false

           read-only ( RO): true


uuid ( RO)                : c46b7a61-9e82-4ea1-88ca-692cd4a9204b

          name-label ( RW): ROOT-97

    name-description ( RW):

             sr-uuid ( RO): 6d4bea51-f253-3b43-2f2f-6d7ba3261ed3

        virtual-size ( RO): 34359738368

            sharable ( RO): false

           read-only ( RO): false


[root@abmpc-hv04 ~]# xe vdi-list name-label=DATA-97

uuid ( RO)                : bc868e3d-b3c0-4c6a-a6fc-910bc4dd1722

          name-label ( RW): DATA-97

    name-description ( RW):

             sr-uuid ( RO): 6d4bea51-f253-3b43-2f2f-6d7ba3261ed3

        virtual-size ( RO): 107374182400

            sharable ( RO): false

           read-only ( RO): false


uuid ( RO)                : a8c187cc-2ba0-4928-8acf-2afc012c036c

          name-label ( RW): DATA-97

    name-description ( RW):

             sr-uuid ( RO): 6d4bea51-f253-3b43-2f2f-6d7ba3261ed3

        virtual-size ( RO): 107374182400

            sharable ( RO): false

           read-only ( RO): true


Following is how I plan to fix the corrupted DB entries. Note: using uuid of VDI volume with
read/write access as the path values:


1. for ROOT-97 volume:

Update volumes set removed=NOW() where id=124;
Update volumes set removed=NULL where id=317;
Update volumes set path=c46b7a61-9e82-4ea1-88ca-692cd4a9204b where id=317;


2) for DATA-97 volume:

Update volumes set pool_id=8 where id=125;

Update volumes set path=bc868e3d-b3c0-4c6a-a6fc-910bc4dd1722 where id=125;


Would this work?


Thanks for all the helps anyone can provide.  I have a total of 4 VM instances with 8 volumes
in this situation need to be fixed.


Yiping
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message