incubator-sling-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Sling > Chunked File Upload Support
Date Wed, 13 Feb 2013 13:06:00 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/2042/9/1/_/styles/combined.css?spaceKey=SLING&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/SLING/Chunked+File+Upload+Support">Chunked
File Upload Support</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~shgupta">Shashank
Gupta</a>
    </h4>
        <br/>
                         <h4>Changes (2)</h4>
                                 
    
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
    
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" > <br>h4. Upload of first and
intermediate chunks <br></td></tr>
            <tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">
<br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">Client
uses PATCH method to upload binary chunk of file.  <br></td></tr>
            <tr><td class="diff-unchanged" >{code:title= Upload the first/intermediate
chunk request} <br>Request: <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
    
            </table>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <div>
<ul>
    <li><span class='TOCOutline'>1</span> <a href='#ChunkedFileUploadSupport-ChunkedFileUpload'>Chunked
File Upload</a></li>
    <li><span class='TOCOutline'>2</span> <a href='#ChunkedFileUploadSupport-UseCases%3A'>Use
Cases:</a></li>
    <li><span class='TOCOutline'>3</span> <a href='#ChunkedFileUploadSupport-Approach'>Approach</a></li>
    <li><span class='TOCOutline'>4</span> <a href='#ChunkedFileUploadSupport-ProtocolSpecification'>Protocol
Specification</a></li>
<ul>
    <li><span class='TOCOutline'>4.1</span> <a href='#ChunkedFileUploadSupport-Uploadoffirstandintermediatechunks'>Upload
of first and intermediate chunks</a></li>
    <li><span class='TOCOutline'>4.2</span> <a href='#ChunkedFileUploadSupport-Uploadoflastchunk'>Upload
of last chunk</a></li>
    <li><span class='TOCOutline'>4.3</span> <a href='#ChunkedFileUploadSupport-Querytoretrievelastsuccessfulchunkupload'>Query
to retrieve last successful chunk upload</a></li>
    <li><span class='TOCOutline'>4.4</span> <a href='#ChunkedFileUploadSupport-Abortchunkedupload'>Abort
chunked upload</a></li>
    <li><span class='TOCOutline'>4.5</span> <a href='#ChunkedFileUploadSupport-Querysling%27scapabilitytosupportchunkedupload'>Query
sling's capability to support chunked upload</a></li>
</ul>
</ul></div>

<h1><a name="ChunkedFileUploadSupport-ChunkedFileUpload"></a>Chunked File
Upload</h1>

<p>Status: DRAFT<br/>
Created: 20. January 2013<br/>
Author: shgupta<br/>
JIRA: <a href="https://issues.apache.org/jira/browse/SLING-2707" class="external-link"
rel="nofollow">SLING-2707</a><br/>
References: - <a href="http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html"
class="external-link" rel="nofollow">http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html</a><br/>
Updated: -</p>

<h1><a name="ChunkedFileUploadSupport-UseCases%3A"></a>Use Cases:</h1>
<p>1. Large file upload - With high speed internet connections, advent of cloud and
HD going mainstream, Sling support large files (&gt; 2GB) upload.<br/>
2. Fault tolerant uploads - Sling provides capability to resume upload from failure point.
It doesn't require client to restart the complete upload process. </p>

<h1><a name="ChunkedFileUploadSupport-Approach"></a>Approach</h1>
<p>Sling provides an endpoint which accepts file chunks in accordance with a specified
protocol. Sling client slices the file in chunks, and upload the chunks in serial manner to
server. Each chunk is numbered serially and the last chunk has additional identifier to distinguish
from the rest chunks. Sling endpoint upon receiving the last chunk, stitches all chunks into
a single file. The file inputstream is passed to inner layers for further processing.</p>

<p>In case of upload failures, sling provides support to query the last chunk uploaded
till failure point. Client resumes chunk upload from last failure point. </p>

<h1><a name="ChunkedFileUploadSupport-ProtocolSpecification"></a>Protocol
Specification </h1>

<h4><a name="ChunkedFileUploadSupport-Uploadoffirstandintermediatechunks"></a>Upload
of first and intermediate chunks</h4>
<p>Client uses PATCH method to upload binary chunk of file. </p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Upload the first/intermediate chunk request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Request:
PATCH /content/dam/dam-folder/catalog.pdf.chunk.&lt;chunk_number&gt;.res HTTP/1.1
Host: localhost:4502
Content-Type:
Date: &lt;date&gt;
Content-Length: 5242880

</pre>
</div></div>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>First/Intermediate chunk upload response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Response:
HTTP/1.1 201 CREATED
Content-Length: 0
Location: /content/dam/dam-folder/catalog.pdf.chunk
Connection: close
Server: localhost:4502
</pre>
</div></div>
<p>The location header provides path to retrieve chunk upload attributes.</p>

<h4><a name="ChunkedFileUploadSupport-Uploadoflastchunk"></a>Upload of last
chunk</h4>
<p>Client appends "/last" to upload url to signify that it would be last that needs
to be uploaded. The last chunk upload request can optionally be multipart request if client
requires to send additional parameters to sling.</p>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Upload last chunk request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Request:
PATCH /content/dam/dam-folder/catalog.pdf.chunk.&lt;chunk_number&gt;.res/last HTTP/1.1
Host: localhost:4502
Date: &lt;date&gt;
Content-type: multipart/form-data, boundary=AaB03x

--AaB03x
content-disposition: form-data; name=<span class="code-quote">"fileSize"</span>

210000000
--AaB03x
content-disposition: form-data; name=<span class="code-quote">"file"</span>; fileName=<span
class="code-quote">"catalog.pdf"</span>
Content-Type: application/pdf
Content-Transfer-Encoding: binary

$binarydata
--AaB03x--

</pre>
</div></div>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Upload last chunk response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Response:
HTTP/1.1 201/CREATED
Content-Length: 0
Location: /content/dam/dam-folder/catalog.pdf
Content-MD5: &lt;md5 checksum of the merged file&gt;
Connection: close
</pre>
</div></div>
<p>The location header provides location of fully uploaded asset.</p>

<h4><a name="ChunkedFileUploadSupport-Querytoretrievelastsuccessfulchunkupload"></a>Query
to retrieve last successful chunk upload</h4>
<p>Returns the number of last chunk upload and bytes uploaded successfully on sling.
</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Get last successful chunk upload request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Request
GET /content/dam/dam-folder/catalog.pdf.chunk.json HTTP/1.1
Host: localhost:4502
Date: &lt;date&gt;
Content-Length: 0
</pre>
</div></div>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Get last successful upload chunk response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Response:
HTTP/1.1 200/OK
Connection: close

{
    <span class="code-quote">"chunkNumber"</span>: 8
    <span class="code-quote">"bytesuploaded"</span>: 19872
}
</pre>
</div></div>

<p>If no broken/discontinued upload sling returns 404 not found response. </p>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Get erroneous last successful upload chunk response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Response:
HTTP/1.1 404/Not Found
</pre>
</div></div>

<h4><a name="ChunkedFileUploadSupport-Abortchunkedupload"></a>Abort chunked
upload</h4>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Abort chunked upload Request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Request
DELETE /content/dam/dam-folder/catalog.pdf.chunk.res HTTP/1.1
Host: localhost:4502
Date: &lt;date&gt;
</pre>
</div></div>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Abort chunked upload response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Response
HTTP/1.1 200/OK
Connection: close
</pre>
</div></div>

<h4><a name="ChunkedFileUploadSupport-Querysling%27scapabilitytosupportchunkedupload"></a>Query
sling's capability to support chunked upload</h4>
<p>Client can use option method to query Sling's capability to support chunked upload.
In response, the Allow header contains PATCH method, indicating that Sling supports chunked
upload on the url. </p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Options chunked upload request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Request
OPTIONS /content/dam/dam-folder/catalog.pdf.chunk.res HTTP/1.1
Host: localhost:4502
Date: &lt;date&gt;
</pre>
</div></div>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Options chunked upload response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Response
HTTP/1.1 200 OK
Connection: Keep-Alive
Allow: OPTIONS, GET,  PATCH
</pre>
</div></div>

    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href="https://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
        </div>
        <a href="https://cwiki.apache.org/confluence/display/SLING/Chunked+File+Upload+Support">View
Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=30749779&revisedVersion=12&originalVersion=11">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/SLING/Chunked+File+Upload+Support?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

Mime
View raw message