Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: pass (athena.apache.org: domain of min.chen@citrix.com
 designates 66.165.176.89 as permitted sender)
From: Min Chen <min.chen@citrix.com>
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
CC: Mike Tutkowski <mike.tutkowski@solidfire.com>
Subject: Re: S3/Swift Problem around Virtual Size
Thread-Topic: S3/Swift Problem around Virtual Size
Thread-Index: AQHPvkkEIQpnpN83xkWjGzXe2mONKpvdlm6AgAABiAD//4t3AA==
Date: Fri, 22 Aug 2014 21:28:23 +0000
Message-ID: <D01D03CA.60E7B%min.chen@citrix.com>
References: 
 <CACOnxChMHL__7o4FBmxzSpKp3qkLwxCOjKBUiiAygpbj0V1dcQ@mail.gmail.com>
 <77B337AF224FD84CBF8401947098DD8731235398@SJCPEX01CL02.citrite.net>
 <CAFgSx9GKe7wv_ZfrS2po4Z_JHetP73ynhd=7Xv1bV=2TPxMjWg@mail.gmail.com>
In-Reply-To: 
 <CAFgSx9GKe7wv_ZfrS2po4Z_JHetP73ynhd=7Xv1bV=2TPxMjWg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/14.4.3.140616
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <5D3286E80032874590D3FF75DE1D1669@citrix.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

No. For S3/Swift, register template will directly upload templates to S3
without going through staging NFS. It will only be copied to staging NFS
when we first use that template to provision a VM.

Thanks
-min

On 8/22/14 2:25 PM, "Francois Gaudreault" <fgaudreault@cloudops.com> wrote:

>Edison,
>
>Isnt the templates downloaded to the Staging NFS first?
>
>FG
>On Aug 22, 2014 5:20 PM, "Edison Su" <Edison.su@citrix.com> wrote:
>
>> I know the reason why the size of template doesn=B9t have correct virtua=
l
>> size if it=B9s registered in S3/Swift:
>> In case of s3/swift, the template is directly stored into s3/swift
>>through
>> swift/s3 api, there is no place for cloudstack to look into template, to
>> find out the virtual size during template registration.
>> While, if secondary storage is NFS, the template is first stored on
>> NFS(which has file system), cloudstack can unzip the template(if it=B9s =
a
>> zipped file), and read virtual size from the file, then report back to
>>mgt
>> server.
>> In order to fix it, we can add some code as: all the templates
>>registered
>> on Swift/S3, need to be downloaded to a NFS intermediate storage before
>>it
>> can be consumed by primary storage. After the download finished, then we
>> check virtual size of template, and report back to mgt server/update DB
>>etc.
>>
>> From: Mike Tutkowski [mailto:mike.tutkowski@solidfire.com]
>> Sent: Friday, August 22, 2014 1:38 PM
>> To: dev@cloudstack.apache.org
>> Cc: Edison Su
>> Subject: S3/Swift Problem around Virtual Size
>>
>> Hi,
>>
>> This was brought up in a different e-mail thread, but I wanted to make
>>it
>> more clear that it's related to CloudStack's download code around
>>S3/Swift,
>> so I'm opening up a new thread.
>>
>> Francois (from CloudOps) noticed today that when he downloaded a
>>template
>> (VHD format) to Swift (but it looks like the same applies for S3) that
>>the
>> physical and virtual sizes are set to be the same.
>>
>> This appears to have the following consequence:
>>
>> You can download a template with a physical size of, say, 3 GB and a
>>root
>> disk that's supposed to be, say, 20 GB. Instead of the virtual size
>>showing
>> as 20 GB, it shows as 3 GB.
>>
>> This is not an issue with NFS. In that situation, the two sizes are
>> correctly accounted for.
>>
>> What later can happen is the template is downloaded from Swift and takes
>> up an unexpected amount of space on the XenServer storage repository
>>(SR).
>>
>> If there is enough space on the SR, this isn't too big of a deal.
>>However,
>> for so-called managed storage plug-ins (examples are SolidFire and
>> CloudByte), this will lead to them dynamically creating a SAN volume of
>>the
>> wrong size.
>>
>> Francois opened up the following ticket:
>>
>> https://issues.apache.org/jira/browse/CLOUDSTACK-7406
>>
>> Thanks!
>>
>> --
>> Mike Tutkowski
>> Senior CloudStack Developer, SolidFire Inc.
>> e: mike.tutkowski@solidfire.com<mailto:mike.tutkowski@solidfire.com>
>> o: 303.746.7302
>> Advancing the way the world uses the cloud<
>> http://solidfire.com/solution/overview/?video=3Dplay>=81
>>