Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: pass (nike.apache.org: domain of min.chen@citrix.com designates
 66.165.176.89 as permitted sender)
From: Min Chen <min.chen@citrix.com>
To: "dev@cloudstack.apache.org" <dev@cloudstack.apache.org>
Subject: Re: Object based Secondary storage.
Thread-Topic: Object based Secondary storage.
Thread-Index: 
 AQHOYc9+DZgqy53xg0mkcPBX+dynC5knS80AgAGNcoCAAFNTgIAAd4YAgAAgDYCAABysAIAAOIeAgAASYQCAAKOqgIAAKdaAgAAHBoCACmjTAIAAEz2AgASOwQCAAB+wgIAAezQA//+LsgA=
Date: Mon, 17 Jun 2013 16:49:31 +0000
Message-ID: <CDE48B96.3F085%min.chen@citrix.com>
In-Reply-To: <C3C4318A-E653-492D-B6C8-5987B6C48399@basho.com>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/14.3.4.130416
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <3173D807954E994C8EEB06CF5B98E0EA@citrix.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

John,

	Let me clarify, we didn't do extra compression before sending to S3. Only
when user provides a URL pointing to a compressed template during
registering, we will just download that template to S3 without
decompressing it afterwards as we did for NFS currently. If the register
url provided user is not compressed format, we will just send uncompressed
version to S3.

	Thanks
	-min

On 6/17/13 9:45 AM, "John Burwell" <jburwell@basho.com> wrote:

>Min,
>
>Why are objects being compressed before being sent to S3?
>
>Thanks,
>-John
>
>On Jun 17, 2013, at 12:24 PM, Min Chen <min.chen@citrix.com> wrote:
>
>> Hi Tom,
>>=20
>> 	Thanks for your testing. Glad to hear that multipart is working fine by
>> using Cloudian. Regarding your questions about .gz template, that
>>behavior
>> is as expected. We will upload it to S3 as its .gz format. Only when the
>> template is used and downloaded to primary storage, we will use staging
>> area to decompress it.
>> 	We will look at the bugs you filed and update them accordingly.
>>=20
>> 	-min
>>=20
>> On 6/17/13 12:31 AM, "Thomas O'Dowd" <tpodowd@cloudian.com> wrote:
>>=20
>>> Thanks Min - I filed 3 small issues today. I've a couple more but I
>>>want
>>> to try and repeat them again before I file them and I've no time right
>>> now. Please let me know if you need any further detail on any of these.
>>>=20
>>> https://issues.apache.org/jira/browse/CLOUDSTACK-3027
>>> https://issues.apache.org/jira/browse/CLOUDSTACK-3028
>>> https://issues.apache.org/jira/browse/CLOUDSTACK-3030
>>>=20
>>> An example of the other issues I'm running into are that when I upload
>>> an .gz template on regular NFS storage, it is automatically
>>>decompressed
>>> for me where as with S3 the template remains as a .gz file. Is this
>>> correct or not? Also, perhaps related but after successfully uploading
>>> the template to S3 and then trying to start an instance using it, I can
>>> select it and go all the way to the last screen where I think the
>>>action
>>> button says launch instance or something and it fails with a resource
>>> unreachable error. I'll have to dig up the error later and file the bug
>>> as my machine got rebooted over the weekend.
>>>=20
>>> The multipart upload looks like it is working correctly though and I
>>>can
>>> verify the checksums etc are correct with what they should be.
>>>=20
>>> Tom.
>>>=20
>>> On Fri, 2013-06-14 at 16:55 +0000, Min Chen wrote:
>>>> HI Tom,
>>>>=20
>>>> 	You can file JIRA ticket for object_store branch by prefixing your
>>>>bug
>>>> with "Object_Store_Refactor" and mentioning that it is using build
>>>>from
>>>> object_store. Here is an example bug filed from Sangeetha against
>>>> object_store branch build:
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-2528.
>>>> 	If you use devcloud for testing, you may run into an issue where ssvm
>>>> cannot access public url when you register a template, so register
>>>> template will fail. You may have to set up internal web server inside
>>>> devcloud and post template to be registered there to give a URL that
>>>> devcloud can access. We mainly used devcloud to run our TestNG
>>>> automation
>>>> test earlier, and then switched to real hypervisor for real testing.
>>>> 	Thanks
>>>> 	-min
>>>>=20
>>>> On 6/14/13 1:46 AM, "Thomas O'Dowd" <tpodowd@cloudian.com> wrote:
>>>>=20
>>>>> Edison,
>>>>>=20
>>>>> I've got devcloud running along with the object_store branch and I've
>>>>> finally been able to test a bit today.
>>>>>=20
>>>>> I found some issues (or things that I think are bugs) and would like
>>>>>to
>>>>> file a few issues. I know where the bug database is and I have an
>>>>> account but what is the best way to file bugs against this particular
>>>>> branch? I guess I can select "Future" as the version? What other way
>>>> are
>>>>> feature branches usually identified in issues? Perhaps in the
>>>>>subject?
>>>>> Please let me know the preference.
>>>>>=20
>>>>> Also, can you describe (or point me at a document) what the best way
>>>>>to
>>>>> test against the object_store branch is? So far I have been doing the
>>>>> following but I'm not sure it is the best?
>>>>>=20
>>>>> a) setup devcloud.
>>>>> b) stop any instances on devcloud from previous runs
>>>>>     xe vm-shutdown --multiple
>>>>> c) check out and update the object_store branch.
>>>>> d) clean build as described in devcloud doc (ADIDD for short)
>>>>> e) deploydb (ADIDD)
>>>>> f) start management console (ADIDD) and wait for it.
>>>>> g) deploysvr (ADIDD) in another shell.
>>>>> h) on devcloud machine use xentop to wait for 2 vms to launch.
>>>>>   (I'm not sure what the nfs vm is used for here??)
>>>>> i) login on gui -> infra -> secondary and remove nfs secondary
>>>>>storage
>>>>> j) add s3 secondary storage (using cache of old secondary storage?)
>>>>>=20
>>>>> Then rest of testing starts from here... (and also perhaps in step j)
>>>>>=20
>>>>> Thanks,
>>>>>=20
>>>>> Tom.
>>>>> --=20
>>>>> Cloudian KK - http://www.cloudian.com/get-started.html
>>>>> Fancy 100TB of full featured S3 Storage?
>>>>> Checkout the Cloudian=AE Community Edition!
>>>>>=20
>>>>=20
>>>=20
>>> --=20
>>> Cloudian KK - http://www.cloudian.com/get-started.html
>>> Fancy 100TB of full featured S3 Storage?
>>> Checkout the Cloudian=AE Community Edition!
>>>=20
>>=20
>