cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-5446) KVM-Secondary Store down-Even after secondary store is brought back up after being down for few hours,snapshot jobs do not get triggered with reason "there is other active snapshot tasks on the instance to which the volume is attached"
Date Wed, 19 Nov 2014 23:54:34 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14218731#comment-14218731
] 

ASF subversion and git services commented on CLOUDSTACK-5446:
-------------------------------------------------------------

Commit 0e3aebbb9db795b1acbc14d4f035da3cb2345f34 in cloudstack's branch refs/heads/master from
Edison Su
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=0e3aebb ]

CLOUDSTACK-5446:
delete all the leftover snapshots on primary storage in case of snapshot
errors, after a new backup snapshot is finished


> KVM-Secondary Store down-Even after secondary store is brought back up after being down
for few hours,snapshot jobs do not get triggered with reason "there is other active snapshot
tasks on the instance to which the volume is attached"
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-5446
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5446
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.3.0
>         Environment: Build from 4.3
>            Reporter: Sangeetha Hariharan
>            Assignee: edison su
>             Fix For: 4.4.0, 4.5.0
>
>         Attachments: agentdown.rar, ssdown.rar
>
>
> KVM - Secondary Store down - Even after secondary store is brought back up after being
down for few hours ,  snapshot jobs do not get triggered with reason "here is other active
snapshot tasks on the instance to which the volume is attached, please try again later"
> Set up:
> Advanced Zone set up with 2 KVM (RHEL 6.3) hosts.
> Steps to reproduce the problem:
> 1. Deploy 5 Vms in each of the hosts with 10 GB ROOT volume size , so we start with 10
Vms.
> 2. Start concurrent snapshots for ROOT volumes of all the Vms.
> 3. Shutdown the Secondary storage server when the snapshots are in the progress.
> 4. Bring the Secondary storage server up after  ~ 12 hours.
> When the secondary storage was down:
> The snapshot jobs that were in progress  timed out after 6 hours.
> Even after this , I do not see the hourly snapshots being scheduled due to the following
reason:
>  
> 2013-12-10 13:07:49,285 WARN  [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-cf0775f7)
Scheduling snapshot failed due to com.cloud.utils.exception.CloudRuntimeException: There is
other active snapshot tasks on the instance to which the volume is attached, please try again
later
> Even after the secondary storage was brought up , there is no hourly snapshots being
scheduled due to the same reason - "com.cloud.utils.exception.CloudRuntimeException: There
is other active snapshot tasks on the instance to which the volume is attached, please try
again later"
> mysql> select * FROM snapshots;
> +----+----------------+------------+-----------+-----------+------------------+------------------+------+-------------------------------------------------+--------------------------------------+---------------+------------------+-------------+---------------------+---------+----------------+----------+------------+--------------+-----------------+---------+-------+
> | id | data_center_id | account_id | domain_id | volume_id | disk_offering_id | status
          | path | name                                            | uuid                
                | snapshot_type | type_description | size        | created             | removed
| backup_snap_id | swift_id | sechost_id | prev_snap_id | hypervisor_type | version | s3_id
|
> +----+----------------+------------+-----------+-----------+------------------+------------------+------+-------------------------------------------------+--------------------------------------+---------------+------------------+-------------+---------------------+---------+----------------+----------+------------+--------------+-----------------+---------+-------+
> |  1 |              1 |          6 |         1 |        45 |               18 | BackedUp
        | NULL | TestVM-tiny-host-0ps-0-0_ROOT-45_20131209234410 | ee2c46b7-7623-439a-9a30-63eec1d95c56
|             3 | HOURLY           | 21474836480 | 2013-12-09 23:44:10 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> |  2 |              1 |          6 |         1 |        43 |               18 | BackedUp
        | NULL | TestVM-1_ROOT-43_20131209234410                 | 62c01389-49df-475b-b225-617d3903d25a
|             3 | HOURLY           | 21474836480 | 2013-12-09 23:44:10 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> |  3 |              1 |          6 |         1 |        54 |               19 | BackedUp
        | NULL | TestVM-tiny-host-1ps-0-4_ROOT-54_20131209234410 | 8c7621f0-5d94-4f22-b3f8-7954c810e1be
|             3 | HOURLY           | 21474836480 | 2013-12-09 23:44:10 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> |  4 |              1 |          6 |         1 |        53 |               19 | BackedUp
        | NULL | TestVM-tiny-host-1ps-0-3_ROOT-53_20131209234410 | 30c72de6-2a3e-4bf2-88da-b2a3b6ae9c66
|             3 | HOURLY           | 21474836480 | 2013-12-09 23:44:10 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> |  5 |              1 |          6 |         1 |        52 |               19 | BackedUp
        | NULL | TestVM-tiny-host-1ps-0-2_ROOT-52_20131209234411 | e8e9f99a-5a8b-4cfd-b743-327925955980
|             3 | HOURLY           | 21474836480 | 2013-12-09 23:44:11 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> |  6 |              1 |          6 |         1 |        51 |               19 | BackedUp
        | NULL | TestVM-tiny-host-1ps-0-1_ROOT-51_20131209234411 | 07e4edc8-b289-4f94-8696-ccf9db60e3ab
|             3 | HOURLY           | 21474836480 | 2013-12-09 23:44:11 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> |  7 |              1 |          6 |         1 |        50 |               19 | BackedUp
        | NULL | TestVM-tiny-host-1ps-0-0_ROOT-50_20131209234411 | b285aa1a-6650-4b90-9c2c-e8bf0c507bfb
|             3 | HOURLY           | 21474836480 | 2013-12-09 23:44:11 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> |  8 |              1 |          6 |         1 |        49 |               18 | BackedUp
        | NULL | TestVM-tiny-host-0ps-0-4_ROOT-49_20131209234412 | 31e37c8d-aab2-4ee6-9c43-c23501b08b02
|             3 | HOURLY           | 21474836480 | 2013-12-09 23:44:12 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> |  9 |              1 |          6 |         1 |        48 |               18 | BackedUp
        | NULL | TestVM-tiny-host-0ps-0-3_ROOT-48_20131209234412 | 60b6023e-138f-42c2-9867-f559e1d95352
|             3 | HOURLY           | 21474836480 | 2013-12-09 23:44:12 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 10 |              1 |          6 |         1 |        47 |               18 | CreatedOnPrimary
| NULL | TestVM-tiny-host-0ps-0-2_ROOT-47_20131209234412 | fa37ebe5-f36c-4f19-b99f-249ab95c4e61
|             3 | HOURLY           | 21474836480 | 2013-12-09 23:44:12 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 11 |              1 |          6 |         1 |        46 |               18 | BackedUp
        | NULL | TestVM-tiny-host-0ps-0-1_ROOT-46_20131209234412 | bd571765-b28e-433e-b480-ccdc3bc0450b
|             3 | HOURLY           | 21474836480 | 2013-12-09 23:44:12 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 12 |              1 |          6 |         1 |        45 |               18 | CreatedOnPrimary
| NULL | TestVM-tiny-host-0ps-0-0_ROOT-45_20131210004410 | 4c6f50c2-661e-4bad-9f07-357b2ca4e7fc
|             3 | HOURLY           | 21474836480 | 2013-12-10 00:44:10 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 13 |              1 |          6 |         1 |        43 |               18 | CreatedOnPrimary
| NULL | TestVM-1_ROOT-43_20131210004410                 | bd3744ae-182f-4a0b-9685-5bef0c747365
|             3 | HOURLY           | 21474836480 | 2013-12-10 00:44:10 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 14 |              1 |          6 |         1 |        54 |               19 | CreatedOnPrimary
| NULL | TestVM-tiny-host-1ps-0-4_ROOT-54_20131210004410 | 14f53a2f-bb4b-40e5-aa90-e7b4be4a1076
|             3 | HOURLY           | 21474836480 | 2013-12-10 00:44:10 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 15 |              1 |          6 |         1 |        53 |               19 | CreatedOnPrimary
| NULL | TestVM-tiny-host-1ps-0-3_ROOT-53_20131210004410 | 88dd82f5-ade5-4dd5-a147-e0a35a15be85
|             3 | HOURLY           | 21474836480 | 2013-12-10 00:44:10 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 16 |              1 |          6 |         1 |        52 |               19 | CreatedOnPrimary
| NULL | TestVM-tiny-host-1ps-0-2_ROOT-52_20131210004410 | 1830af7d-97e5-488c-bc37-39c11961e509
|             3 | HOURLY           | 21474836480 | 2013-12-10 00:44:10 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 17 |              1 |          6 |         1 |        51 |               19 | CreatedOnPrimary
| NULL | TestVM-tiny-host-1ps-0-1_ROOT-51_20131210004410 | 3d452970-f042-4af6-a8cc-3003482da9a8
|             3 | HOURLY           | 21474836480 | 2013-12-10 00:44:10 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 18 |              1 |          6 |         1 |        50 |               19 | CreatedOnPrimary
| NULL | TestVM-tiny-host-1ps-0-0_ROOT-50_20131210004410 | 0cb1b176-bcfe-4bc2-9b5e-07ec8e9e97f5
|             3 | HOURLY           | 21474836480 | 2013-12-10 00:44:10 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 19 |              1 |          6 |         1 |        49 |               18 | CreatedOnPrimary
| NULL | TestVM-tiny-host-0ps-0-4_ROOT-49_20131210004411 | 87337d8b-e698-47b1-961a-32479d44a6e5
|             3 | HOURLY           | 21474836480 | 2013-12-10 00:44:11 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 20 |              1 |          6 |         1 |        48 |               18 | CreatedOnPrimary
| NULL | TestVM-tiny-host-0ps-0-3_ROOT-48_20131210004411 | c7b321bd-31b4-4c9c-be8f-c9231cb08530
|             3 | HOURLY           | 21474836480 | 2013-12-10 00:44:11 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 21 |              1 |          6 |         1 |        47 |               18 | CreatedOnPrimary
| NULL | TestVM-tiny-host-0ps-0-2_ROOT-47_20131210004411 | ea1a71b0-9b52-42e5-a252-7a3e8938bc0f
|             3 | HOURLY           | 21474836480 | 2013-12-10 00:44:11 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 22 |              1 |          6 |         1 |        46 |               18 | CreatedOnPrimary
| NULL | TestVM-tiny-host-0ps-0-1_ROOT-46_20131210004411 | f1fbc899-5f55-472c-b798-639350b2facd
|             3 | HOURLY           | 21474836480 | 2013-12-10 00:44:11 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 33 |              1 |          6 |         1 |        45 |               18 | CreatedOnPrimary
| NULL | TestVM-tiny-host-0ps-0-0_ROOT-45_20131210022749 | 396c80a8-299c-42b2-8c3d-32da5af9536c
|             3 | HOURLY           | 21474836480 | 2013-12-10 02:27:49 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 34 |              1 |          6 |         1 |        43 |               18 | CreatedOnPrimary
| NULL | TestVM-1_ROOT-43_20131210022749                 | 9444a355-08a6-4c7d-98f2-74729a0b28b5
|             3 | HOURLY           | 21474836480 | 2013-12-10 02:27:49 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 35 |              1 |          6 |         1 |        54 |               19 | CreatedOnPrimary
| NULL | TestVM-tiny-host-1ps-0-4_ROOT-54_20131210022749 | 09cbfc03-7f9f-470f-bc34-e2bbad72b9ba
|             3 | HOURLY           | 21474836480 | 2013-12-10 02:27:49 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 36 |              1 |          6 |         1 |        53 |               19 | CreatedOnPrimary
| NULL | TestVM-tiny-host-1ps-0-3_ROOT-53_20131210022749 | 2e7259f0-82d2-4910-9047-617d54bda035
|             3 | HOURLY           | 21474836480 | 2013-12-10 02:27:49 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 37 |              1 |          6 |         1 |        52 |               19 | CreatedOnPrimary
| NULL | TestVM-tiny-host-1ps-0-2_ROOT-52_20131210022749 | 420230ed-a037-4ed3-a019-031d24fd0bfb
|             3 | HOURLY           | 21474836480 | 2013-12-10 02:27:49 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 38 |              1 |          6 |         1 |        51 |               19 | CreatedOnPrimary
| NULL | TestVM-tiny-host-1ps-0-1_ROOT-51_20131210022750 | 6bf87e88-3a14-467f-ac5b-ab620fd03206
|             3 | HOURLY           | 21474836480 | 2013-12-10 02:27:50 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 39 |              1 |          6 |         1 |        50 |               19 | CreatedOnPrimary
| NULL | TestVM-tiny-host-1ps-0-0_ROOT-50_20131210022750 | 93021350-9df1-4234-9df2-869d9e9a912f
|             3 | HOURLY           | 21474836480 | 2013-12-10 02:27:50 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 40 |              1 |          6 |         1 |        49 |               18 | CreatedOnPrimary
| NULL | TestVM-tiny-host-0ps-0-4_ROOT-49_20131210022750 | 4699ebdf-3093-4f2f-86b7-ae5c9195cc6a
|             3 | HOURLY           | 21474836480 | 2013-12-10 02:27:50 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 41 |              1 |          6 |         1 |        48 |               18 | CreatedOnPrimary
| NULL | TestVM-tiny-host-0ps-0-3_ROOT-48_20131210022750 | 8d663979-31a2-48ad-884b-f1ef85a1f4d5
|             3 | HOURLY           | 21474836480 | 2013-12-10 02:27:50 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> | 42 |              1 |          6 |         1 |        46 |               18 | CreatedOnPrimary
| NULL | TestVM-tiny-host-0ps-0-1_ROOT-46_20131210022751 | f80c2994-b612-4234-add4-27d323cf3dd3
|             3 | HOURLY           | 21474836480 | 2013-12-10 02:27:51 | NULL    | NULL  
        |     NULL |       NULL |         NULL | KVM             | 2.2     |  NULL |
> +----+----------------+------------+-----------+-----------+------------------+------------------+------+-------------------------------------------------+--------------------------------------+---------------+------------------+-------------+---------------------+---------+----------------+----------+------------+--------------+-----------------+---------+-------+
> 32 rows in set (0.00 sec)
> mysql>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message