From users-return-26199-apmail-cloudstack-users-archive=cloudstack.apache.org@cloudstack.apache.org Tue Aug 9 00:37:43 2016 Return-Path: X-Original-To: apmail-cloudstack-users-archive@www.apache.org Delivered-To: apmail-cloudstack-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5EE9619FE0 for ; Tue, 9 Aug 2016 00:37:43 +0000 (UTC) Received: (qmail 34312 invoked by uid 500); 9 Aug 2016 00:37:42 -0000 Delivered-To: apmail-cloudstack-users-archive@cloudstack.apache.org Received: (qmail 34261 invoked by uid 500); 9 Aug 2016 00:37:42 -0000 Mailing-List: contact users-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@cloudstack.apache.org Delivered-To: mailing list users@cloudstack.apache.org Received: (qmail 34244 invoked by uid 99); 9 Aug 2016 00:37:42 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Aug 2016 00:37:42 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 97508C6169 for ; Tue, 9 Aug 2016 00:37:41 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.802 X-Spam-Level: X-Spam-Status: No, score=-0.802 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 2V4v33VCsxqx for ; Tue, 9 Aug 2016 00:37:40 +0000 (UTC) Received: from mail-pf0-f171.google.com (mail-pf0-f171.google.com [209.85.192.171]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTPS id D0D935F230 for ; Tue, 9 Aug 2016 00:37:39 +0000 (UTC) Received: by mail-pf0-f171.google.com with SMTP id p64so124144406pfb.1 for ; Mon, 08 Aug 2016 17:37:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=CwSdJh7mG9Wa6BJHpqX1frmn5xqsoD6i7Op5mXkgBxY=; b=eLa+hPpxb/UKKhw1Rck6wDUtcZnsH52o5ZfTBnf3xVBl18E6i94gDi/vnlKKBXQnnl 0oQed9zk6wujlF0bsTTx4GaNvD7Y2ZK3lLgcSwBRXeP2ua2GMpm1GOvl99J+GrnfwhB8 lyYBESJYvBmqM5FeIgON8LxW8PiibddR5USR3qL71LT4m01T7T/Z/ilJ5jC+ZVc+18Qv k/0iHc4Jl0Y8vo2Jwt9mbLM+uCYT+2err0K48xEusFQa8nXz5sgp5wvNpRzcwlj5UNM3 9GHr8PhgnGSuRNf9NGkfmFozqCc+jxT5QDAUO7n5NdB/Gx8VCBPRwXvRguYjX5ITOJWE G4Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=CwSdJh7mG9Wa6BJHpqX1frmn5xqsoD6i7Op5mXkgBxY=; b=OovDtJgSs8dWZR+p1HXMWRVM+bFVD+d/1X67kqmLCzpplXE/dhiUP7/j02QylxJrbz PJEreh5TRctWl+WeqaAtVF2b64IokQBKzsaybQ+KGUfXSVT3PNdDIqLrLn7inOP4RpGL /gm5O6UlpHmNVVMbfpOE1uvMgE1LNBvaLBp7nUBxo+UWOZCRVyZDVtNRYwG3XJyFUa3k GcFAy7jOHVMm0jr/11UkNe+gdRJel+QAHOGcvxbRApCixpgc3FjrAM9mR7ZnIru9COrA stR2DLfitFLszAiAN7p/9f9uX2dl6P8wiABgeKhX+nCedK1eFUfr/f17LjssQGTKCylj Afag== X-Gm-Message-State: AEkooutkLsAJem92CvBna3gzr1CpIMgfwMA9Gnw2Gx9WxUit5s9MasiYOV1A2C9nPA8WQA== X-Received: by 10.98.111.133 with SMTP id k127mr165792177pfc.90.1470703052072; Mon, 08 Aug 2016 17:37:32 -0700 (PDT) Received: from [0.0.0.0] (dev1.cloudsand.com. [162.243.147.22]) by smtp.gmail.com with ESMTPSA id l82sm50785789pfk.8.2016.08.08.17.37.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Aug 2016 17:37:31 -0700 (PDT) Subject: Re: Mess after volume migration. To: users@cloudstack.apache.org References: From: ilya Message-ID: <6358ca79-1e19-2082-6a7a-b7d45051cc25@gmail.com> Date: Mon, 8 Aug 2016 17:37:30 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit this happened to us on non XEN hypervisor as well. CloudStack has a timeout for a long running jobs - which i assume in your case - it has exceeded. Changing volumes table should be enough by referencing proper pool_id. Just make sure that data size matches on both ends. consider changing "copy.volume.wait" (if that does not help) also "vm.job.timeout" Regards ilya On 8/8/16 3:54 AM, Makrand wrote: > Guys, > > My setup:- ACS 4.4.2. Hypervisor: XENserver 6.2. > > I tried moving a volume in running VM from primary storage A to primary > storage B (using GUI of cloudstack). Please note, primary storage A LUN > (LUN7)is coming out of one storage box and primary storage B LUN (LUN14) > is from another. > > For VM1 with 250GB data volume (51 GB used space), I was able to move this > volume without any glitch in about 26mins. > > But for VM2 with 250Gb data volume (182 GB used space), the migration > continued for about ~110 mins and then failed with follwing exception in > very end with message like:- > > 2016-08-06 14:30:57,481 WARN [c.c.h.x.r.CitrixResourceBase] > (DirectAgent-192:ctx-5716ad6d) Task failed! Task record: > uuid: 308a8326-2622-e4c5-2019-3beb > 87b0d183 > nameLabel: Async.VDI.pool_migrate > nameDescription: > allowedOperations: [] > currentOperations: {} > created: Sat Aug 06 12:36:27 UTC 2016 > finished: Sat Aug 06 14:30:32 UTC 2016 > status: failure > residentOn: com.xensource.xenapi.Host@f242d3ca > progress: 1.0 > type: > result: > errorInfo: [SR_BACKEND_FAILURE_80, , Failed to mark VDI hidden > [opterr=SR 96e879bf-93aa-47ca-e2d5-e595afbab294: error aborting existing > process]] > otherConfig: {} > subtaskOf: com.xensource.xenapi.Task@aaf13f6f > subtasks: [] > > > So cloudstack just removed the JOB telling it failed, says the mangement > server log. > > A) But when I am checking it at hyeprvisor level, the volume is on new SR > i.e. on LUN14. Strange huh? So now the new uuid for this volume from XE cli > is like > > [root@gcx-bom-compute1 ~]# xe vbd-list > vm-uuid=3fcb3070-e373-3cf9-d0aa-0a657142a38d > uuid ( RO) : f15dc54a-3868-8de8-5427-314e341879c6 > vm-uuid ( RO): 3fcb3070-e373-3cf9-d0aa-0a657142a38d > vm-name-label ( RO): i-22-803-VM > vdi-uuid ( RO): cc1f8e83-f224-44b7-9359-282a1c1e3db1 > empty ( RO): false > device ( RO): hdb > > B) But luckily I had the entry taken before migration and it shows like:- > > uuid ( RO) : f15dc54a-3868-8de8-5427-314e341879c6 > vm-uuid ( RO): 3fcb3070-e373-3cf9-d0aa-0a657142a38d > vm-name-label ( RO): i-22-803-VM > vdi-uuid ( RO): 7c073522-a077-41a0-b9a7-7b61847d413b > empty ( RO): false > device ( RO): hdb > > C) Since this failed at cloudstack, the DB is still holding old value. > Here is current volume table entry in DB > > id: 1004 >> account_id: 22 >> domain_id: 15 >> pool_id: 18 >> last_pool_id: NULL >> instance_id: 803 >> device_id: 1 >> name: >> cloudx_globalcloudxchange_com_W2797T2808S3112_V1462960751 >> uuid: a8f01042-d0de-4496-98fa-a0b13648bef7 >> size: 268435456000 >> folder: NULL >> path: 7c073522-a077-41a0-b9a7-7b61847d413b >> pod_id: NULL >> data_center_id: 2 >> iscsi_name: NULL >> host_ip: NULL >> volume_type: DATADISK >> pool_type: NULL >> disk_offering_id: 6 >> template_id: NULL >> first_snapshot_backup_uuid: NULL >> recreatable: 0 >> created: 2016-05-11 09:59:12 >> attached: 2016-05-11 09:59:21 >> updated: 2016-08-06 14:30:57 >> removed: NULL >> state: Ready >> chain_info: NULL >> update_count: 42 >> disk_type: NULL >> vm_snapshot_chain_size: NULL >> iso_id: NULL >> display_volume: 1 >> format: VHD >> min_iops: NULL >> max_iops: NULL >> hv_ss_reserve: 0 >> 1 row in set (0.00 sec) >> > > > So the path variable shows value as 7c073522-a077-41a0-b9a7-7b61847d413b > and pool id as 18. > > The VM is running as of now, but I am sure the moment I will reboot, this > volume will be gone or worst VM won't boot. This is production VM BTW. > > D) So I think I need to edit volume table for path and pool_id parameters > and need to place new values in place and then reboot VM. Do I need to make > any more changes in DB in some other tables for same? Any comment/help is > much appreciated. > > > > > -- > Best, > Makrand >