cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Min Chen (JIRA)" <>
Subject [jira] [Commented] (CLOUDSTACK-3027) Object_Store_Refactor - Uploaded template S3 content-type is not appropriate.
Date Wed, 04 Dec 2013 19:10:35 GMT


Min Chen commented on CLOUDSTACK-3027:

Hi Thomas, what you have seen is expected behavior. As I mentioned in this bug, the fix is
using HEAD content type returned from HttpClient.GetMethod invoked on the URL given for the
template to determine what content-type I should send before sending to S3. In your case,
the template 1 (system vm template) is automatically downloaded to S3 when you register S3
as secondary storage. I tried 

localhost:cloud-asf minc$ curl -i -X HEAD
HTTP/1.1 200 OK
x-amz-id-2: dm/UlDs7TOFrxJoe7S6wGN5p1a4PuaHgv59/2UH3uySdhZ8S0iYs4ZXD9s+PlfIo
x-amz-request-id: BDF70CC804E094C5
Date: Wed, 04 Dec 2013 18:57:18 GMT
x-amz-meta-cb-modifiedtime: Mon, 05 Aug 2013 06:50:05 GMT
Last-Modified: Mon, 05 Aug 2013 06:51:43 GMT
ETag: "5e8f4deb290e1fbd2585fee7ded761fe-27"
Accept-Ranges: bytes
Content-Type: application/octet-stream
Content-Length: 223003018
Server: AmazonS3

As you can see, application/octet-stream is given by the server which hosts this template

For template 2, it is downloaded to S3 during template sync when secondary storage is up and
running. The URL in the DB for this template is
If you run curl HEAD command on this url, you will see that its content-type returned is text/plain,
which may be wrong, but we have no control on that. For your template 3, it seems that your
server which hosts ttylinux_pv.vhd returns "application/octet-stream" as its content-type,
so we can set correctly. 

> Object_Store_Refactor - Uploaded template S3 content-type is not appropriate.
> -----------------------------------------------------------------------------
>                 Key: CLOUDSTACK-3027
>                 URL:
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Template
>    Affects Versions: 4.2.0
>         Environment: latest object_store branch on fedora 17
> devcloud on same machine
> Cloudian (for S3 services) on separate machine. (expect similar result with other S3
object stores).
>            Reporter: Thomas O'Dowd
>            Assignee: Min Chen
>             Fix For: 4.3.0
> This bug is related to the object_store branch.
> Steps:
> 1. setup S3 object storage (can be amazon)
> 2. Add S3 as secondary storage
> 3. Upload a new template (I uploaded "tinyLinux.vhd.gz") by giving a url on my local
network where I had it hosted.
> 4. The template will be uploaded to S3.
> Next using s3cmd or other tool, have a look at the Content-Type of the object that is
stored by issuing either a HEAD or a GET request on that object.
> Here is what I got when i captured the HEAD request using ngrep.
> =================== HEAD request and response ===================
> HEAD /template/tmpl/2/201/201-2-f9a12429-7cf4-3df5-b81c-420f09c1bbcd/tinyLinux.vhd.gz
> Host:
> Accept-Encoding: identity.
> Date: Mon, 17 Jun 2013 05:06:19 GMT.
> Content-Length: 0.
> Authorization: AWS 00d25034c817eeb8c095:g/S63k9LHeZfz73l+moWm7MJ7z0=.
> User-Agent: Boto/2.9.4 (linux2).
> .
> ##
> T -> [AP]
> HTTP/1.1 200 OK.
> Date: Mon, 17 Jun 2013 05:06:19 GMT.
> x-amz-request-id: A55A3220D70B11E2.
> Last-Modified: Fri, 14 Jun 2013 07:05:27 GMT.
> ETag: "5b5c3506aef231519d0434f9749c951d-5".
> Content-Type: application/x-www-form-urlencoded; charset=utf-8.
> Content-Length: 25712307.
> =================== end of HEAD request and response =================
> Notice in the HTTP response that the "Content-Type" header is set to "application/x-www-form-urlencoded;
charset=utf-8". This does not correctly describe the template content type.
> The Content-Type was set by the client (Cloudstack) in the "Multipart Upload Initiate"
request at the beginning of the upload and should be fixed at that point. I haven't looked
at the code but perhaps we need to set a header or add another parameter when initiating the
multipart upload.
> Of course, this begs the question... well what is the correct content type? I looked
around to see if I could find one for .vhd files but with no luck. Perhaps "application/octet-stream"
or even "application/x-gzip" (as this particular object is compressed).

This message was sent by Atlassian JIRA

View raw message