incubator-sling-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Sling > Chunked File Upload Support
Date Tue, 22 Jan 2013 12:06:00 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/2042/9/1/_/styles/combined.css?spaceKey=SLING&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/SLING/Chunked+File+Upload+Support">Chunked
File Upload Support</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~shgupta">Shashank
Gupta</a>
    </h4>
        <br/>
                         <h4>Changes (8)</h4>
                                 
    
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
    
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >{code:title= Upload the first/intermediate
chunks} <br>Request: <br></td></tr>
            <tr><td class="diff-changed-lines" >POST <span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">/content/dam/dam-folder/asset.pdf.multipartupload.html/chunk/&lt;chunk_number&gt;</span>
<span class="diff-added-words"style="background-color: #dfd;">/content/dam/dam-folder/asset.pdf.chunkupload.html/chunk/&lt;chunk_number&gt;</span>
HTTP/1.1 <br></td></tr>
            <tr><td class="diff-unchanged" >Host: localhost:4502 <br>Content-Type:
<br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >{code:title= Upload Last chunk request}
<br>Request: <br></td></tr>
            <tr><td class="diff-changed-lines" >POST <span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">/content/dam/dam-folder/asset.pdf.multipartupload.html/parts/&lt;part_number&gt;/last</span>
<span class="diff-added-words"style="background-color: #dfd;">/content/dam/dam-folder/asset.pdf.chunkupload.html/parts/&lt;part_number&gt;/last</span>
HTTP/1.1 <br></td></tr>
            <tr><td class="diff-unchanged" >Host: localhost:4502 <br>Date:
&lt;date&gt; <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >Response: <br>HTTP/1.1 201/CREATED
<br></td></tr>
            <tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">Location:
http://localhost:4502/content/dam/dam-folder/asset.pdf <br></td></tr>
            <tr><td class="diff-unchanged" >Content-Length: 0 <br>Content-MD5:
&lt;md5 checksum of the merged file&gt; <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >{code:title= Get last successful chunk
upload request} <br>Request <br></td></tr>
            <tr><td class="diff-changed-lines" >GET <span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">/content/dam/dam-folder/asset.pdf.multipartupload.html</span>
<span class="diff-added-words"style="background-color: #dfd;">/content/dam/dam-folder/asset.pdf.chunkupload.html</span>
HTTP/1.1 <br></td></tr>
            <tr><td class="diff-unchanged" >Host: localhost:4502 <br>Date:
&lt;date&gt; <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >Response: <br>HTTP/1.1 200/OK
<br></td></tr>
            <tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">Location:
http://localhost:4502/content/dam/dam-folder/asset.pdf <br></td></tr>
            <tr><td class="diff-unchanged" >Connection: close <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >{code:title= Get erroneous last successful
upload chunk response} <br>Response: <br></td></tr>
            <tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">HTTP/1.1
500/OK <br>Location: http://localhost:4502/content/dam/dam-folder/asset.pdf <br>Connection:
close <br> <br>{ <br>    &quot;error&quot;: no upload found <br>}
<br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">HTTP/1.1
404/Not Found <br></td></tr>
            <tr><td class="diff-unchanged" >{code} <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >Response <br>HTTP/1.1 200/OK
<br></td></tr>
            <tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">Location:
http://localhost:4502/content/dam/dam-folder/asset.pdf <br></td></tr>
            <tr><td class="diff-unchanged" >Connection: close <br>{code}
<br></td></tr>
    
            </table>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <div>
<ul>
    <li><span class='TOCOutline'>1</span> <a href='#ChunkedFileUploadSupport-ChunkedFileUpload'>Chunked
File Upload</a></li>
    <li><span class='TOCOutline'>2</span> <a href='#ChunkedFileUploadSupport-UseCases%3A'>Use
Cases:</a></li>
    <li><span class='TOCOutline'>3</span> <a href='#ChunkedFileUploadSupport-Approach'>Approach</a></li>
    <li><span class='TOCOutline'>4</span> <a href='#ChunkedFileUploadSupport-ProtocolSpecification'>Protocol
Specification</a></li>
<ul>
    <li><span class='TOCOutline'>4.1</span> <a href='#ChunkedFileUploadSupport-Uploadoffirstandintermediatechunks'>Upload
of first and intermediate chunks</a></li>
    <li><span class='TOCOutline'>4.2</span> <a href='#ChunkedFileUploadSupport-Uploadoflastchunk'>Upload
of last chunk</a></li>
    <li><span class='TOCOutline'>4.3</span> <a href='#ChunkedFileUploadSupport-Querytoretrievelastsuccessfulchunkupload'>Query
to retrieve last successful chunk upload</a></li>
    <li><span class='TOCOutline'>4.4</span> <a href='#ChunkedFileUploadSupport-Abortchunkedupload'>Abort
chunked upload</a></li>
</ul>
</ul></div>

<h1><a name="ChunkedFileUploadSupport-ChunkedFileUpload"></a>Chunked File
Upload</h1>

<p>Status: DRAFT<br/>
Created: 20. January 2013<br/>
Author: shgupta<br/>
JIRA: <a href="https://issues.apache.org/jira/browse/SLING-2707" class="external-link"
rel="nofollow">SLING-2707</a><br/>
References: - <a href="http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html"
class="external-link" rel="nofollow">http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html</a><br/>
Updated: -</p>

<h1><a name="ChunkedFileUploadSupport-UseCases%3A"></a>Use Cases:</h1>
<p>1. Large file upload - With high speed internet connections, advent of cloud and
HD going mainstream, Sling support large files (&gt; 2GB) upload.<br/>
2. Fault tolerant uploads - Sling provides capability to resume upload from failure point.
It doesn't require client to restart the complete upload process. </p>

<h1><a name="ChunkedFileUploadSupport-Approach"></a>Approach</h1>
<p>Sling provides an endpoint which accepts file chunks in accordance with a specified
protocol. Sling client slices the file in chunks, and upload the chunks in serial manner to
server. Each chunk is numbered serially and the last chunk has additional identifier to distinguish
from the rest chunks. Sling endpoint upon receiving the last chunk, stitches all chunks into
a single file. The file inputstream is passed to inner layers for further processing.</p>

<p>In case of upload failures, sling provides support to query the last chunk uploaded
till failure point. Client resumes chunk upload from last failure point. </p>

<h1><a name="ChunkedFileUploadSupport-ProtocolSpecification"></a>Protocol
Specification </h1>

<h4><a name="ChunkedFileUploadSupport-Uploadoffirstandintermediatechunks"></a>Upload
of first and intermediate chunks</h4>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Upload the first/intermediate chunks</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Request:
POST /content/dam/dam-folder/asset.pdf.chunkupload.html/chunk/&lt;chunk_number&gt;
HTTP/1.1
Host: localhost:4502
Content-Type:
Date: &lt;date&gt;
Content-Length: 5242880

</pre>
</div></div>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>First/Intermediate chunkUpload Response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Response:
HTTP/1.1 200 OK
Content-Length: 0
Connection: close
Server: localhost:4502
</pre>
</div></div>

<h4><a name="ChunkedFileUploadSupport-Uploadoflastchunk"></a>Upload of last
chunk</h4>
<p>Client appends "/last" to upload url to signify that it would be last that needs
to be uploaded. The last chunk upload request can optionally be multipart request if client
requires to pass additional parameters. </p>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Upload Last chunk request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Request:
POST /content/dam/dam-folder/asset.pdf.chunkupload.html/parts/&lt;part_number&gt;/last
HTTP/1.1
Host: localhost:4502
Date: &lt;date&gt;
Content-type: multipart/form-data, boundary=AaB03x

--AaB03x
content-disposition: form-data; name=<span class="code-quote">"fileSize"</span>

210000000
--AaB03x
content-disposition: form-data; name=<span class="code-quote">"file"</span>; fileName=<span
class="code-quote">"asset.pdf"</span>
Content-Type: application/pdf
Content-Transfer-Encoding: binary

$binarydata
--AaB03x--

</pre>
</div></div>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Upload Last chunk Response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Response:
HTTP/1.1 201/CREATED
Content-Length: 0
Content-MD5: &lt;md5 checksum of the merged file&gt;
Connection: close
</pre>
</div></div>

<h4><a name="ChunkedFileUploadSupport-Querytoretrievelastsuccessfulchunkupload"></a>Query
to retrieve last successful chunk upload</h4>
<p>Returns the number of last chunk upload successfully on sling. </p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Get last successful chunk upload request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Request
GET /content/dam/dam-folder/asset.pdf.chunkupload.html HTTP/1.1
Host: localhost:4502
Date: &lt;date&gt;
Content-Length: 0
</pre>
</div></div>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Get last successful upload chunk response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Response:
HTTP/1.1 200/OK
Connection: close

{
    <span class="code-quote">"chunkNumber"</span>: 8
}
</pre>
</div></div>

<p>If no broken/discontinued upload sling returns 500 error </p>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Get erroneous last successful upload chunk response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Response:
HTTP/1.1 404/Not Found
</pre>
</div></div>

<h4><a name="ChunkedFileUploadSupport-Abortchunkedupload"></a>Abort chunked
upload</h4>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Abort chunked upload Request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Request
DELETE /content/dam/dam-folder/asset.pdf.multipartupload.html HTTP/1.1
Host: localhost:4502
Date: &lt;date&gt;
</pre>
</div></div>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Abort chunked Upload Response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
Response
HTTP/1.1 200/OK
Connection: close
</pre>
</div></div>
    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href="https://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
        </div>
        <a href="https://cwiki.apache.org/confluence/display/SLING/Chunked+File+Upload+Support">View
Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=30749779&revisedVersion=3&originalVersion=2">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/SLING/Chunked+File+Upload+Support?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

Mime
View raw message