From users-return-31776-archive-asf-public=cust-asf.ponee.io@cloudstack.apache.org Thu Nov 22 21:18:18 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 77A1C180645 for ; Thu, 22 Nov 2018 21:18:17 +0100 (CET) Received: (qmail 88683 invoked by uid 500); 22 Nov 2018 20:18:16 -0000 Mailing-List: contact users-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@cloudstack.apache.org Delivered-To: mailing list users@cloudstack.apache.org Received: (qmail 88638 invoked by uid 99); 22 Nov 2018 20:18:15 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 22 Nov 2018 20:18:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 4B58618CE54 for ; Thu, 22 Nov 2018 20:18:15 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.407 X-Spam-Level: X-Spam-Status: No, score=-1.407 tagged_above=-999 required=6.31 tests=[DKIMWL_WL_MED=-1.458, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id a1f4kW0AAQH9 for ; Thu, 22 Nov 2018 20:18:13 +0000 (UTC) Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id A6A2D610C1 for ; Thu, 22 Nov 2018 20:18:12 +0000 (UTC) Received: by mail-pg1-f173.google.com with SMTP id d72so2111020pga.9 for ; Thu, 22 Nov 2018 12:18:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=dwm4T7FcjkUW3CFRNLQHIe1Lu4Kf0IrgUC/Obcj4hXk=; b=RYuMsg8xnhOGkZkDoHrA6fmVhn2s1KBJdDWA70EZbS1XHZeOG7jVrIVvDikYqaZk4f HCxlllEz0d7G/R91/GjLXGojKBEazJChCFqN9ecXhQNZPPZO3ipdGrdkO9AJra9hdv6c W6zx+U2qdpuICTAwxR/C5NR1HsnCJxxI0ytjS0cqhqJBPHiZ+VMCWPUXy08FqPqRAoiF 4Ljj47gY570BH1B4YHTZfgj9Qb03I3ujK6Uhk/jHuGEaWEnp384uF/Dfe48TaEMV7t0O WqzhwauyZrDClC/VusRG58Eiz57b88/0+MV0yIEnJ4Oro0C1xUNgwGheZ77r5WmTbZSn awzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=dwm4T7FcjkUW3CFRNLQHIe1Lu4Kf0IrgUC/Obcj4hXk=; b=fhJ2F3EtjQ20sfWUBSk9Y410sf1jdMWouGQOZPXJeD/8JrRas5Ns4rIdN/lz7pZo59 9stKwm99Xsyv9a5QF1BSq+Mg6UGr+SHFGmDzg9nkygXCf1QefiKuZ9ixKfsP+eWa/bSC A3g0CuQs0mgIkWdeR8ouBbYWTNtwcr22Es3r5ayjqcnuE+iIcAOunV8edzN/jn98MrEL 3A0nfBozCz+Pv0m/NSeLWj7GM/MkWYnsAPOCmW2F5MbSjYQTA52UpKfmlDCFo0x90Y6D NB8z33SLjhqF/TQO3JUqrXPDfjDDxOfxcf4TiPwkQDC8LBJ2a9b6fmk+3A4DLAtpqfrS z9BQ== X-Gm-Message-State: AA+aEWayezl3olRIHSFZJcilKFDHVur8l4+WMhYbfKdvfCj21G00S+/H DS9y7ZjXgIMbCGEXcYGlIrQcrk+f X-Google-Smtp-Source: AFSGD/VusL96qvHiOzOt7vXBqdeCN4AG2NiH1o4Ar1o0YBGAfcSURSmkPO4zcmtWUvPszU1Ju7wMcw== X-Received: by 2002:a63:4b60:: with SMTP id k32mr11245579pgl.186.1542917885722; Thu, 22 Nov 2018 12:18:05 -0800 (PST) Received: from [172.19.20.134] ([208.91.115.31]) by smtp.gmail.com with ESMTPSA id l11sm6972445pff.65.2018.11.22.12.18.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Nov 2018 12:18:05 -0800 (PST) Subject: Re: KVM NFS template image To: users@cloudstack.apache.org References: <01f1ed04-bfe1-9526-b1a2-59db10960f41@gmail.com> <1D8636F9-FB53-4E4A-A7FB-CAE7269A11BF@shapeblue.com> From: ran huang Message-ID: <8c4bf072-f67a-873b-237b-a6815236c241@gmail.com> Date: Thu, 22 Nov 2018 12:18:04 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Thanks Andrija, just tested myself with expunge and works as expected. However, for KVM, when I revert a qcow disk from snapshot, which removes the backing chain to template, the template will not be removed. So it seems like despite the qcow disk is no longer backed by the template, the template will still consider the disk as its children in this case(revert from snapshot). regards, Ran On 11/19/2018 10:43 AM, Andrija Panic wrote: > new template, deployed new VM, destroyed VM (with Exunge option)... > > up to 120sec later... (storage.cleanup.interval=120 sec, global config > option) > > 2018-11-19 19:35:59,525 DEBUGStorage pool garbage collector found 1 > templates to clean up in storage pool: Primary-storage - NFS > 2018-11-19 19:35:59,525 DEBUG [c.c.s.StorageManagerImpl] > (StorageManager-Scavenger-1:ctx-2c88c8e0) (logid:040f4ad1) Storage pool > garbage collector has marked template with ID: 219 on pool 4 for garbage > collection > > Another 120sec later... (storage.cleanup.delay=120sec) > > 2018-11-19 19:37:59,598 DEBUG [c.c.s.StorageManagerImpl] > (StorageManager-Scavenger-2:ctx-f9dd338d) (logid:9ae40975) Storage pool > garbage collector found 1 templates to clean up in storage pool: > Primary-storage - NFS > ... > 2018-11-19 19:37:59,638 DEBUG [c.c.t.TemplateManagerImpl] > (StorageManager-Scavenger-2:ctx-f9dd338d) (logid:9ae40975) Evicting > TmplPool[37-219-4-563ea0f5-5164-4ac4-a183-728f418269b7] > 2018-11-19 19:37:59,643 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru] > (StorageManager-Scavenger-2:ctx-f9dd338d) (logid:9ae40975) > getCommandHostDelegation: class com.cloud.agent.api.storage.DestroyCommand > ... > 2018-11-19 19:37:59,665 DEBUG [c.c.t.TemplateManagerImpl] > (StorageManager-Scavenger-2:ctx-f9dd338d) (logid:9ae40975) Successfully > evicted template andrija-test-tmpl from storage pool null > > template "andrija-test-tmpl" deleted... > > Hope that helps Run. > > Cheers > Andrija > > On Mon, 19 Nov 2018 at 19:11, Andrija Panic wrote: > >> True (at least I'm sure for SolidFire) - but I believe in general also >> (will test this now...) >> >> On Mon, 19 Nov 2018 at 18:51, Dag Sonstebo >> wrote: >> >>> Developers please correct me... but as far as I remember there is a >>> garbage collector which does remove the templates from primary storage once >>> they are not needed (i.e. have no more "child VMs"). This is controlled by >>> the global setting "storage.template.cleanup.enabled". >>> >>> Regards, >>> Dag Sonstebo >>> Cloud Architect >>> ShapeBlue >>> >>> >>> On 16/11/2018, 22:51, "ran huang" wrote: >>> >>> Hi Andrija, >>> >>> Thanks for the clarification and quick response >>> >>> regards, >>> Ran >>> >>> On 11/16/2018 02:15 PM, Andrija Panic wrote: >>> > Hi Ran, >>> > >>> > templates stays on Primary Storage "forever", at least for NFS >>> (they are >>> > moved from Secondary to Primary when you deploy a very first VM from >>> > specific template). All VMs have this templates qcow2 as baking >>> (parent) >>> > image. >>> > >>> > This template is a qcow2 copy of a file from Secondary Storage - >>> and is >>> > considered a "parent" image, from which all child images (VM >>> volumes) are >>> > created - as you stated baking file (qcow linked clones, in official >>> > terminology) >>> > >>> > you can have i.e. 100 VMs all linking (having it's backing file...) >>> to a >>> > template qcow2 file. >>> > So in other words, it's not supposed to be removed. >>> > >>> > Does this make sense? >>> > >>> > Cheers >>> > >>> > >>> > >>> >>> Dag.Sonstebo@shapeblue.com >>> www.shapeblue.com >>> Amadeus House, Floral Street, London WC2E 9DPUK >>> @shapeblue >>> >>> >>> >>>> On Fri, 16 Nov 2018 at 22:38, ran huang wrote: >>> > >>> >> Greetings All, >>> >> >>> >> For qcow2 format images, when creating a new VM in KVM, the >>> template >>> >> image is copied from secondary storage to primary storage, and the >>> root >>> >> volume image is created with the template image as a backing file. >>> >> >>> >> But when I break this backing chain on primary(expunge VM or >>> revert to a >>> >> snapshot previously created on the root volume image), the template >>> >> image is not deleted. >>> >> >>> >> Might I ask how is the template image going to be cleaned from the >>> >> primary storage? >>> >> >>> >> >>> >> addendum: >>> >> CS ver 4.9.2 on CentOS 7.2 >>> >> >>> >> regards, >>> >> Ran >>> >> >>> > >>> >>> >>> >>> >> -- >> >> Andrija Panić >> >