From dev-return-110523-apmail-cloudstack-dev-archive=cloudstack.apache.org@cloudstack.apache.org Tue Jan 9 21:45:27 2018 Return-Path: X-Original-To: apmail-cloudstack-dev-archive@www.apache.org Delivered-To: apmail-cloudstack-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4D8721749F for ; Tue, 9 Jan 2018 21:45:27 +0000 (UTC) Received: (qmail 51848 invoked by uid 500); 9 Jan 2018 21:45:26 -0000 Delivered-To: apmail-cloudstack-dev-archive@cloudstack.apache.org Received: (qmail 51800 invoked by uid 500); 9 Jan 2018 21:45:26 -0000 Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list dev@cloudstack.apache.org Received: (qmail 51743 invoked by uid 99); 9 Jan 2018 21:45:26 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Jan 2018 21:45:26 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 0A040C1CF1 for ; Tue, 9 Jan 2018 21:45:26 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.89 X-Spam-Level: * X-Spam-Status: No, score=1.89 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, KAM_SHORT=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=cloudops.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id Il9BteJG72zZ for ; Tue, 9 Jan 2018 21:45:24 +0000 (UTC) Received: from mail-vk0-f54.google.com (mail-vk0-f54.google.com [209.85.213.54]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 48F325F36C for ; Tue, 9 Jan 2018 21:45:24 +0000 (UTC) Received: by mail-vk0-f54.google.com with SMTP id o73so10338875vkd.2 for ; Tue, 09 Jan 2018 13:45:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudops.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=cjyPE2NhyrgabbaQzyosJW6peENq04bWLqyn/MwQeIg=; b=Zajfvj6jfwb9nf3Cy5Ex6TuCev3kE5YmDTTvbc2J+1ZetgKVzyPtMK4yQunIuWalgb JuTt/W+/PQVVTV2Zrgb3pFke0+DhAiSt3RvzpFA+rXHr8RwLAFiV44wPRWknGR5B4odc bfibvT8w5f2kig80JdB2T7joSKnpWDwRp+rxY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=cjyPE2NhyrgabbaQzyosJW6peENq04bWLqyn/MwQeIg=; b=CllwAcHOvqxQg0nTQHqQCZDEmTTt0TQUHxkg989dc5Hfhk12tWD3XMzxCt+QqFO6Hl Jo8GixPGvnGx4esNTA33xoHiXWlx1oFL5ukjLAT3feZueMNGXAvFpZR1Hey7bfcdvtXV tPx/KyVh1yahpkkdunUhuhYV7UEV9Vv49880WtHziDfV6YvYdrJn5WE6l4lIP5kjyyxf Yxnb+EkXfVitbBho5QJ52GjrLKNMNOufOMYRLQTlTiV6toM/QGBuKLUBWTT0hAaExRY9 T9XnnCcLAObrudKIjLizyA+1Z36yj0YU531yRxLgU0HYqsQuGMJIsNXqSUAB1cpEtNfH X8wQ== X-Gm-Message-State: AKwxytcrHNjvSQzaW4OlWJlsCfLAXWOtFbKAxWsmroY1AWKg5F+q2ZvG jar1FXYDmh3ci52GQKojmVQKuLUjj4AfOMcgiDem9tY= X-Google-Smtp-Source: ACJfBovfdDXEH/cB8GpOaRJMlQDEo6yyy16GLOcUWrchc/WUTLvJQvsPyS/C6DZkrTLMLTv5+RbMDM6TuelgEz38zxI= X-Received: by 10.31.63.147 with SMTP id m141mr13455010vka.193.1515534322853; Tue, 09 Jan 2018 13:45:22 -0800 (PST) MIME-Version: 1.0 Received: by 10.159.49.139 with HTTP; Tue, 9 Jan 2018 13:44:52 -0800 (PST) In-Reply-To: References: From: Khosrow Moossavi Date: Tue, 9 Jan 2018 16:44:52 -0500 Message-ID: Subject: Re: Squeeze another PR (#2398) in 4.11 milestone To: dev@cloudstack.apache.org Content-Type: multipart/alternative; boundary="001a114dbbbaf57f3a05625ed4b6" --001a114dbbbaf57f3a05625ed4b6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable That is correct Mike. The quoted part above was misleading, it should have been "at any given point in time *when transaction finished*" Removal of "other" or "current failed" snapshot happens at the very end of the method. The state of SR throughout time would be something like: 1) snapshot-01 (at rest) 2) snapshot-01, snapshot-02 (while taking snapshot-02 on primary storage and sending to secondary storage) 3) snapshot-02 (at rest again, after successful) OR 3) snapshot-01 (at rest again, after failure) Khosrow Moossavi Cloud Infrastructure Developer t 514.447.3456 On Tue, Jan 9, 2018 at 4:33 PM, Tutkowski, Mike wrote: > =E2=80=9Ctechnically we should only have "one" on primary storage at any = given > point in time=E2=80=9D > > I just wanted to follow up on this one. > > When we are copying a delta from the previous snapshot, we should actuall= y > have two snapshots on primary storage for a time. > > If the delta copy is successful, then we delete the older snapshot. If th= e > delta copy fails, then we delete the newest snapshot. > > Is that correct? > > > On Jan 9, 2018, at 11:36 AM, Khosrow Moossavi > wrote: > > > > "We are already deleting snapshots in the primary storage, but we alway= s > > leave behind the last one" > > > > This issue doesn't happen only when something fails. We are not deletin= g > the > > snapshots from primary storage (not on XenServer 6.25+ and not since Fe= b > > 2017) > > > > The fix of this PR is: > > > > 1) when transferred successfully to secondary storage everything except > > "this" > > snapshot get removed (technically we should only have "one" on primary > > storage > > at any given point in time) [towards the end of try block] > > 2) when transferring to secondary storage fails, only "this" in-progres= s > > snapshot > > gets deleted. [finally block] > > > > > > > > On Tue, Jan 9, 2018 at 1:01 PM, Rafael Weing=C3=A4rtner < > > rafaelweingartner@gmail.com> wrote: > > > >> Khosrow, I have seen this issue as well. It happens when there are > problems > >> to transfer the snapshot from the primary to the secondary storage. > >> However, we need to clarify one thing. We are already deleting > snapshots in > >> the primary storage, but we always leave behind the last one. The > problem > >> is that if an error happens, during the transfer of the VHD from the > >> primary to the secondary storage. The failed snapshot VDI is left > behind in > >> primary storage (for XenServer). These failed snapshots can accumulate > with > >> time and cause the problem you described because XenServer will not be > able > >> to coalesce the VHD files of the VM. Therefore, what you are addressin= g > in > >> this PR are cases when an exception happens during the transfer from > >> primary to secondary storage. > >> > >> On Tue, Jan 9, 2018 at 3:25 PM, Khosrow Moossavi < > kmoossavi@cloudops.com> > >> wrote: > >> > >>> Hi community > >>> > >>> We've found [1] and fixed [2] an issue in 4.10 regarding snapshots > >>> remaining on primary storage (XenServer + Swift) which causes VDI cha= in > >>> gets full after some time and user cannot take another snapshot. > >>> > >>> Please include this in 4.11 milestone if you see fit. > >>> > >>> [1]: https://issues.apache.org/jira/browse/CLOUDSTACK-10222 > >>> [2]: https://github.com/apache/cloudstack/pull/2398 > >>> > >>> Thanks > >>> Khosrow > >>> > >> > >> > >> > >> -- > >> Rafael Weing=C3=A4rtner > >> > --001a114dbbbaf57f3a05625ed4b6--