incubator-sling-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Sling > Chunked File Upload Support
Date Mon, 06 May 2013 15:06:00 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/2042/9/1/_/styles/combined.css?spaceKey=SLING&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/SLING/Chunked+File+Upload+Support">Chunked
File Upload Support</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~shgupta">Shashank
Gupta</a>
    </h4>
        <br/>
                         <h4>Changes (3)</h4>
                                 
    
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
    
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >Chunks are stored in within the  actual
path in mixin node type. The content model to store chunk is defined as follows: <br>
<br></td></tr>
            <tr><td class="diff-changed-lines" >{code:title= Content model to
<span class="diff-deleted-words"style="color:#999;background-color:#fdd;text-decoration:line-through;">save</span>
<span class="diff-added-words"style="background-color: #dfd;">store</span> chunk}
<br></td></tr>
            <tr><td class="diff-unchanged" >// node type to store chunk <br>//
offset: offset of chunk in file <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >  - sling:chunksLength (long) <br>
 + * (sling:chunkNode) multiple <br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">{code}
<br></td></tr>
            <tr><td class="diff-unchanged" > <br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">The
typical nt:file node under chunked upload would look like  <br>{code:title= Typical
nt:file nod under chunked upload} <br>/content/dam/folder/sample.pdf [nt:file] <br>
                               + jcr:content [nt:resource] [sling:chunkMixin] <br> 
                                  - jcr:data = empty until completed <br>          
                         - sling:fileLength = 982145 // (filename@Length from client) <br>
								- sling:chunksLength = 30000 //cumulative length of all uploaded chunks <br>
                                   + chunk_0-9999 [sling:chunkNode] <br>           
                                - sling:offset <br>                                
           - jcr:data <br>                                    + chunk_10000-19999 [sling:PartialBinary]
<br>                                    + .... <br></td></tr>
            <tr><td class="diff-unchanged" >{code} <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
    
            </table>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <div>
<ul>
    <li><span class='TOCOutline'>1</span> <a href='#ChunkedFileUploadSupport-ChunkedFileUpload'>Chunked
File Upload</a></li>
    <li><span class='TOCOutline'>2</span> <a href='#ChunkedFileUploadSupport-UseCases%3A'>Use
Cases:</a></li>
    <li><span class='TOCOutline'>3</span> <a href='#ChunkedFileUploadSupport-Approach'>Approach</a></li>
    <li><span class='TOCOutline'>4</span> <a href='#ChunkedFileUploadSupport-ContentModel'>Content
Model</a></li>
    <li><span class='TOCOutline'>5</span> <a href='#ChunkedFileUploadSupport-ProtocolSpecification'>Protocol
Specification</a></li>
<ul>
    <li><span class='TOCOutline'>5.1</span> <a href='#ChunkedFileUploadSupport-UploadchunkusingPOST'>Upload
chunk using POST</a></li>
    <li><span class='TOCOutline'>5.2</span> <a href='#ChunkedFileUploadSupport-QuerySlingabouttheinterruptedchunkuploadstatus'>Query
Sling about the interrupted chunk upload status</a></li>
    <li><span class='TOCOutline'>5.3</span> <a href='#ChunkedFileUploadSupport-Abortchunkupload'>Abort
chunk upload</a></li>
</ul>
    <li><span class='TOCOutline'>6</span> <a href='#ChunkedFileUploadSupport-Implementation'>Implementation</a></li>
<ul>
    <li><span class='TOCOutline'>6.1</span> <a href='#ChunkedFileUploadSupport-ExtensiontoSlingPostServlet'>Extension
to SlingPostServlet</a></li>
    <li><span class='TOCOutline'>6.2</span> <a href='#ChunkedFileUploadSupport-ChunksStorage'>Chunks
Storage</a></li>
    <li><span class='TOCOutline'>6.3</span> <a href='#ChunkedFileUploadSupport-Chunkprocessing'>Chunk
processing</a></li>
</ul>
</ul></div>

<h1><a name="ChunkedFileUploadSupport-ChunkedFileUpload"></a>Chunked File
Upload</h1>

<p>Status: DRAFT<br/>
Created: 20. January 2013<br/>
Author: shgupta<br/>
JIRA: <a href="https://issues.apache.org/jira/browse/SLING-2707" class="external-link"
rel="nofollow">SLING-2707</a><br/>
References: - <a href="http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html"
class="external-link" rel="nofollow">http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html</a><br/>
Updated: -</p>

<h1><a name="ChunkedFileUploadSupport-UseCases%3A"></a>Use Cases:</h1>
<p>1. Large file upload - With high speed internet connections, advent of cloud and
HD going mainstream, Sling support large files (&gt; 2GB) upload.<br/>
2. Fault tolerant uploads - Sling provides capability to resume upload from failure point.
It doesn't require client to restart the complete upload process. </p>

<h1><a name="ChunkedFileUploadSupport-Approach"></a>Approach</h1>
<p>Sling provides an extension to <a href="http://sling.apache.org/site/manipulating-content-the-slingpostservlet-servletspost.html"
class="external-link" rel="nofollow">SlingPostServlet</a> which accepts file chunks
in accordance with a specified protocol. Sling client slices the file in chunks, and upload
the chunks in serial manner to server. Each chunk has "Offset" attribute which identify chunk
position in complete file. <a href="http://sling.apache.org/site/manipulating-content-the-slingpostservlet-servletspost.html"
class="external-link" rel="nofollow">SlingPostServlet</a>upon receiving the last
chunk, stitches all chunks into a single file and save them to the final destination. </p>

<p>In case of upload failures, sling provides support to query the last chunk uploaded
till failure point. Client resumes chunk upload from last failure point. </p>

<h1><a name="ChunkedFileUploadSupport-ContentModel"></a>Content Model</h1>
<p>Chunks are stored in within the  actual path in mixin node type. The content model
to store chunk is defined as follows:</p>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Content model to store chunk</b></div><div
class="codeContent panelContent">
<pre class="code-java">
<span class="code-comment">// node type to store chunk
</span><span class="code-comment">// offset: offset of chunk in file
</span><span class="code-comment">// jcr:data: binary of chunk
</span>[sling:chunkNode] &gt; nt:hierarchyNode
  primaryitem jcr:data
  - sling:offset  (<span class="code-object">long</span>) mandatory
  - jcr:data (binary) mandatory
 
 <span class="code-comment">//-----------------------------------------------------------------------------

</span> <span class="code-comment">// Mixin node type to identify that a node
has chunks
</span> <span class="code-comment">// sling:fileLength : length of complete file
</span> <span class="code-comment">// sling:chunksLength: cumulative length of
all uploaded chunks
</span>[sling:chunkMixin]
  mixin
  - sling:fileLength (<span class="code-object">long</span>)
  - sling:chunksLength (<span class="code-object">long</span>)
  + * (sling:chunkNode) multiple
</pre>
</div></div>

<p>The typical nt:file node under chunked upload would look like </p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Typical nt:file nod under chunked upload</b></div><div
class="codeContent panelContent">
<pre class="code-java">
/content/dam/folder/sample.pdf [nt:file]
                                + jcr:content [nt:resource] [sling:chunkMixin]
                                    - jcr:data = empty until completed
                                    - sling:fileLength = 982145 <span class="code-comment">//
(filename@Length from client)
</span>									- sling:chunksLength = 30000 <span class="code-comment">//cumulative
length of all uploaded chunks
</span>                                    + chunk_0-9999 [sling:chunkNode]
                                            - sling:offset
                                            - jcr:data
                                    + chunk_10000-19999 [sling:PartialBinary]
                                    + ....
</pre>
</div></div>

<h1><a name="ChunkedFileUploadSupport-ProtocolSpecification"></a>Protocol
Specification </h1>


<h3><a name="ChunkedFileUploadSupport-UploadchunkusingPOST"></a>Upload chunk
using POST </h3>

<h4><a name="ChunkedFileUploadSupport-Firstchunkupload"></a>First chunk
upload</h4>
<p>Client uses POST method to parent path to upload binary chunk of file. Client passes
":chunkNumber" and "fileName" as multipart request parameters.<br/>
[request]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>First chunk upload request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
POST /content/dam HTTP/1.1
Authorization: Basic YWRtaW46YWRtaW4=
Transfer-Encoding: chunked
Content-Type: multipart/form-data; boundary=v0_zWJbImtVo0uK-bJWayWUiT3zTBIj6
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Host: localhost:4502

--v0_zWJbImtVo0uK-bJWayWUiT3zTBIj6
Content-Disposition: form-data; name=<span class="code-quote">":chunkNumber"</span>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

1
--v0_zWJbImtVo0uK-bJWayWUiT3zTBIj6
Content-Disposition: form-data; name=<span class="code-quote">"*"</span>; filename=<span
class="code-quote">"catalog.pdf"</span>
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary

$binarydata
</pre>
</div></div>

<p>[response]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>First chunk upload response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
HTTP/1.1 200 OK
Connection: Keep-Alive
Server: Day-Servlet-Engine/4.1.42
Content-Type: text/html;charset=UTF-8
Date: Thu, 28 Feb 2013 14:14:24 GMT
Transfer-Encoding: chunked

&lt;html&gt;
&lt;head&gt;
    &lt;title&gt;Content modified /tmp/sling/chunks&lt;/title&gt;
&lt;/head&gt;
    &lt;body&gt;
    &lt;h1&gt;Content modified /tmp/sling/chunks&lt;/h1&gt;
    &lt;table&gt;
        &lt;tbody&gt;
            &lt;tr&gt;
                &lt;td&gt;Status&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Status"</span>&gt;200&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Message&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Message"</span>&gt;OK&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Location&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>
id=<span class="code-quote">"Location"</span>&gt;/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Parent Location&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"/tmp/sling/chunks"</span>
id=<span class="code-quote">"ParentLocation"</span>&gt;/tmp/sling/chunks&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Path&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Path"</span>&gt;/tmp/sling/chunks&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Referer&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"" id="</span>Referer"&gt;&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;ChangeLog&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"ChangeLog"</span>&gt;&amp;lt;pre&amp;gt;&amp;lt;/pre&amp;gt;&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
        &lt;/tbody&gt;
    &lt;/table&gt;
    &lt;p&gt;&lt;a href=""&gt;Go Back&lt;/a&gt;&lt;/p&gt;
    &lt;p&gt;&lt;a href=<span class="code-quote">"/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>&gt;Modified
Resource&lt;/a&gt;&lt;/p&gt;
    &lt;p&gt;&lt;a href=<span class="code-quote">"/tmp/sling/chunks"</span>&gt;Parent
of Modified Resource&lt;/a&gt;&lt;/p&gt;
    &lt;/body&gt;
&lt;/html&gt;
</pre>
</div></div>
<p>The response shows that chunk is stored in tmp location. <b>The location tag
contains unique chunk upload id which need to be passed on subsequent request.</b></p>

<h4><a name="ChunkedFileUploadSupport-Intermediatechunkupload"></a>Intermediate
chunk upload </h4>
<p>Client require to additionally  pass chunk upload id. <br/>
[request]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Intermediate chunk upload request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
POST /content/dam HTTP/1.1
Authorization: Basic YWRtaW46YWRtaW4=
Transfer-Encoding: chunked
Content-Type: multipart/form-data; boundary=F1MYdmO-MYq-wLW2P2tWIu-7KUH3-r
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Host: localhost:4502

--F1MYdmO-MYq-wLW2P2tWIu-7KUH3-r
Content-Disposition: form-data; name=<span class="code-quote">":chunkNumber"</span>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

3
--F1MYdmO-MYq-wLW2P2tWIu-7KUH3-r
Content-Disposition: form-data; name=<span class="code-quote">":chunkUploadId"</span>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08
--F1MYdmO-MYq-wLW2P2tWIu-7KUH3-r
Content-Disposition: form-data; name=<span class="code-quote">"*"</span>; filename=<span
class="code-quote">"catalog.pdf"</span>
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary

$binarydata
</pre>
</div></div>

<p>[response]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Intermediate chunk upload response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
HTTP/1.1 200 OK
Connection: Keep-Alive
Server: Day-Servlet-Engine/4.1.42
Content-Type: text/html;charset=UTF-8
Date: Thu, 28 Feb 2013 14:14:25 GMT
Transfer-Encoding: chunked

&lt;html&gt;
&lt;head&gt;
    &lt;title&gt;Content modified /tmp/sling/chunks&lt;/title&gt;
&lt;/head&gt;
    &lt;body&gt;
    &lt;h1&gt;Content modified /tmp/sling/chunks&lt;/h1&gt;
    &lt;table&gt;
        &lt;tbody&gt;
            &lt;tr&gt;
                &lt;td&gt;Status&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Status"</span>&gt;200&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Message&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Message"</span>&gt;OK&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Location&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>
id=<span class="code-quote">"Location"</span>&gt;/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Parent Location&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"/tmp/sling/chunks"</span>
id=<span class="code-quote">"ParentLocation"</span>&gt;/tmp/sling/chunks&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Path&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Path"</span>&gt;/tmp/sling/chunks&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Referer&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"" id="</span>Referer"&gt;&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;ChangeLog&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"ChangeLog"</span>&gt;&amp;lt;pre&amp;gt;&amp;lt;/pre&amp;gt;&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
        &lt;/tbody&gt;
    &lt;/table&gt;
    &lt;p&gt;&lt;a href=""&gt;Go Back&lt;/a&gt;&lt;/p&gt;
    &lt;p&gt;&lt;a href=<span class="code-quote">"/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>&gt;Modified
Resource&lt;/a&gt;&lt;/p&gt;
    &lt;p&gt;&lt;a href=<span class="code-quote">"/tmp/sling/chunks"</span>&gt;Parent
of Modified Resource&lt;/a&gt;&lt;/p&gt;
    &lt;/body&gt;
&lt;/html&gt;
</pre>
</div></div>


<h4><a name="ChunkedFileUploadSupport-Lastchunkupload"></a>Last chunk upload</h4>
<p>To indicate the last chunk, client additionally pass ":lastChunk" to true in request
parameter. In last chunk upload response, the changelog has all the modification made in actual
path repository. It would the same response as if client has uploaded file in single shot
to <a href="http://sling.apache.org/site/manipulating-content-the-slingpostservlet-servletspost.html"
class="external-link" rel="nofollow">SlingPostServlet</a><br/>
[request]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Last chunk upload request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
POST /content/dam HTTP/1.1
Authorization: Basic YWRtaW46YWRtaW4=
Transfer-Encoding: chunked
Content-Type: multipart/form-data; boundary=XHnea0Y5RY92mJwpnKXHfl0dfhcqOCe
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Host: localhost:4502

--XHnea0Y5RY92mJwpnKXHfl0dfhcqOCe
Content-Disposition: form-data; name=<span class="code-quote">":lastChunk"</span>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

<span class="code-keyword">true</span>
--XHnea0Y5RY92mJwpnKXHfl0dfhcqOCe
Content-Disposition: form-data; name=<span class="code-quote">":chunkNumber"</span>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

6
--XHnea0Y5RY92mJwpnKXHfl0dfhcqOCe
Content-Disposition: form-data; name=<span class="code-quote">":chunkUploadId"</span>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08
--XHnea0Y5RY92mJwpnKXHfl0dfhcqOCe
Content-Disposition: form-data; name=<span class="code-quote">"*"</span>; filename=<span
class="code-quote">"catalog.pdf"</span>
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary


$binarydata
</pre>
</div></div>
<p>[response]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Last chunk upload response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
HTTP/1.1 200 OK
Connection: Keep-Alive
Server: Day-Servlet-Engine/4.1.42
Content-Type: text/html;charset=UTF-8
Date: Thu, 28 Feb 2013 14:14:25 GMT
Transfer-Encoding: chunked

&lt;html&gt;
&lt;head&gt;
    &lt;title&gt;Content modified /content/dam&lt;/title&gt;
&lt;/head&gt;
    &lt;body&gt;
    &lt;h1&gt;Content modified /content/dam&lt;/h1&gt;
    &lt;table&gt;
        &lt;tbody&gt;
            &lt;tr&gt;
                &lt;td&gt;Status&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Status"</span>&gt;200&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Message&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Message"</span>&gt;OK&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Location&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"/content/dam"</span>
id=<span class="code-quote">"Location"</span>&gt;/content/dam&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Parent Location&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"/content"</span>
id=<span class="code-quote">"ParentLocation"</span>&gt;/content&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Path&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Path"</span>&gt;/content/dam&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Referer&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"" id="</span>Referer"&gt;&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;ChangeLog&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"ChangeLog"</span>&gt;&amp;lt;pre&amp;gt;created(<span
class="code-quote">"/content/dam/catalog.pdf"</span>);&amp;lt;br/&amp;gt;created(<span
class="code-quote">"/content/dam/catalog.pdf/jcr:content"</span>);&amp;lt;br/&amp;gt;modified(<span
class="code-quote">"/content/dam/catalog.pdf/jcr:content/jcr:lastModified"</span>);&amp;lt;br/&amp;gt;modified(<span
class="code-quote">"/content/dam/catalog.pdf/jcr:content/jcr:mimeType"</span>);&amp;lt;br/&amp;gt;modified(<span
class="code-quote">"/content/dam/catalog.pdf/jcr:content/jcr:data"</span>);&amp;lt;br/&amp;gt;&amp;lt;/pre&amp;gt;&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
        &lt;/tbody&gt;
    &lt;/table&gt;
    &lt;p&gt;&lt;a href=""&gt;Go Back&lt;/a&gt;&lt;/p&gt;
    &lt;p&gt;&lt;a href=<span class="code-quote">"/content/dam"</span>&gt;Modified
Resource&lt;/a&gt;&lt;/p&gt;
    &lt;p&gt;&lt;a href=<span class="code-quote">"/content"</span>&gt;Parent
of Modified Resource&lt;/a&gt;&lt;/p&gt;
    &lt;/body&gt;
&lt;/html
</pre>
</div></div>
<p>The location header provides location of fully uploaded asset.The response contains
the $md5_hash of merged binary. </p>

<h3><a name="ChunkedFileUploadSupport-QuerySlingabouttheinterruptedchunkuploadstatus"></a>Query
Sling about the interrupted chunk upload status</h3>
<p>Client sent get request on upload to retrieve chunk upload status.<br/>
[request]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Query interrupted chunk upload request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
GET /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08.json HTTP/1.1
Authorization: Basic YWRtaW46YWRtaW4=
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Host: localhost:4502
</pre>
</div></div>
<p>[response]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Query interrupted chunk upload response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
HTTP/1.1 200 OK
Connection: Keep-Alive
Server: Day-Servlet-Engine/4.1.42
Content-Type: application/json;charset=UTF-8
Content-Length: 155
Date: Thu, 28 Feb 2013 14:14:25 GMT

{<span class="code-quote">"chunksUploaded"</span>:5,<span class="code-quote">"bytesUploaded"</span>:1600,<span
class="code-quote">"jcr:createdBy"</span>:<span class="code-quote">"admin"</span>,<span
class="code-quote">"jcr:created"</span>:<span class="code-quote">"Thu Feb 28
2013 19:44:24 GMT+0530"</span>,<span class="code-quote">"jcr:primaryType"</span>:<span
class="code-quote">"sling:OrderedFolder"</span>}
</pre>
</div></div>
<p>The above response means that 5 continuous chunks with size 1600 (index 0 to index
1599) are successfully uploaded to server. Client should start with :chunkNumber 6 and slice
file from offset 1600.</p>

<h4><a name="ChunkedFileUploadSupport-Ifnobroken%2Fdiscontinueduploadslingreturns404notfoundresponse."></a>If
no broken/discontinued upload sling returns 404 not found response. </h4>
<p>[response]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Chunk upload not found response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
HTTP/1.1 404 Not Found
Connection: Close
Server: Day-Servlet-Engine/4.1.42
Content-Type: text/html;charset=utf-8
Date: Thu, 28 Feb 2013 14:14:25 GMT
Transfer-Encoding: chunked

&lt;!DOCTYPE HTML PUBLIC <span class="code-quote">"-<span class="code-comment">//IETF//DTD
HTML 2.0//EN"</span>&gt;
</span>&lt;html&gt;
    &lt;head&gt;&lt;title&gt;404 No resource found&lt;/title&gt;&lt;/head&gt;
    &lt;body&gt;
        &lt;h1&gt;No resource found&lt;/h1&gt;
        &lt;p&gt;Cannot serve request to /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08
in /libs/sling/servlet/errorhandler/404.jsp&lt;/p&gt;
        
        &lt;h3&gt;Request Progress:&lt;/h3&gt;
&lt;pre&gt;
      0 (2013-02-28 19:44:25) TIMER_START{Request Processing}
      0 (2013-02-28 19:44:25) COMMENT timer_end format is {&amp;lt;elapsed msec&amp;gt;,&amp;lt;timer
name&amp;gt;} &amp;lt;optional message&amp;gt;
      0 (2013-02-28 19:44:25) LOG Method=GET, PathInfo=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08
      0 (2013-02-28 19:44:25) TIMER_START{ResourceResolution}
      0 (2013-02-28 19:44:25) TIMER_END{0,ResourceResolution} URI=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08
resolves to Resource=, type=sling:nonexisting, path=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08,
resource=[NonExistingResource, path=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08]
      0 (2013-02-28 19:44:25) LOG Resource Path Info: SlingRequestPathInfo: path='/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08',
selectorString='<span class="code-keyword">null</span>', extension='<span class="code-keyword">null</span>',
suffix='<span class="code-keyword">null</span>'
      0 (2013-02-28 19:44:25) TIMER_START{ServletResolution}
      0 (2013-02-28 19:44:25) TIMER_START{resolveServlet(, type=sling:nonexisting, path=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08,
resource=[NonExistingResource, path=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08])}
      0 (2013-02-28 19:44:25) LOG {0}: no servlet found
      0 (2013-02-28 19:44:25) TIMER_END{0,resolveServlet(, type=sling:nonexisting, path=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08,
resource=[NonExistingResource, path=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08])}
Using servlet org.apache.sling.servlets.get.DefaultGetServlet
      0 (2013-02-28 19:44:25) TIMER_END{0,ServletResolution} URI=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08
handled by Servlet=org.apache.sling.servlets.get.DefaultGetServlet
      0 (2013-02-28 19:44:25) LOG Applying Requestfilters
      0 (2013-02-28 19:44:25) LOG Calling filter: org.apache.sling.bgservlets.impl.BackgroundServletStarterFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: org.apache.sling.i18n.impl.I18NFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: org.apache.sling.rewriter.impl.RewriterFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.designimporter.CanvasPageDeleteRequestFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.core.impl.WCMRequestFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.adobe.granite.optout.impl.OptOutFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.theme.impl.ThemeResolverFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.foundation.forms.impl.FormsHandlingServlet
      0 (2013-02-28 19:44:25) LOG Calling filter: org.apache.sling.engine.impl.debug.RequestProgressTrackerLogFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.mobile.core.impl.redirect.RedirectFilter
      0 (2013-02-28 19:44:25) LOG RedirectFilter did not redirect (request extension does
not match)
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.core.impl.warp.TimeWarpFilter
      0 (2013-02-28 19:44:25) LOG Applying Componentfilters
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.core.impl.WCMComponentFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.core.impl.WCMDebugFilter
      0 (2013-02-28 19:44:25) TIMER_START{org.apache.sling.servlets.get.DefaultGetServlet#0}
      0 (2013-02-28 19:44:25) TIMER_END{0,org.apache.sling.servlets.get.DefaultGetServlet#0}
      0 (2013-02-28 19:44:25) LOG Applying Error filters
      0 (2013-02-28 19:44:25) LOG Calling filter: org.apache.sling.rewriter.impl.RewriterFilter
      0 (2013-02-28 19:44:25) TIMER_START{handleError:status=404}
      0 (2013-02-28 19:44:25) TIMER_END{0,handleError:status=404} Using handler /libs/sling/servlet/errorhandler/404.jsp
      0 (2013-02-28 19:44:25) LOG Found processor <span class="code-keyword">for</span>
post processing ProcessorConfiguration: {contentTypes=[text/html],order=-1, active=<span
class="code-keyword">true</span>, valid=<span class="code-keyword">true</span>,
processErrorResponse=<span class="code-keyword">true</span>, pipeline=(generator=Config(type=htmlparser,
config={}), transformers=(Config(type=linkchecker, config={}), Config(type=mobile, config=org.apache.sling.jcr.resource.JcrPropertyMap@42e95cbf),
Config(type=mobiledebug, config=org.apache.sling.jcr.resource.JcrPropertyMap@66c821ea), Config(type=contentsync,
config=org.apache.sling.jcr.resource.JcrPropertyMap@5d2abe5d), serializer=Config(type=htmlwriter,
config={}))}
      0 (2013-02-28 19:44:25) TIMER_END{0,Request Processing} Dumping SlingRequestProgressTracker
Entries
&lt;/pre&gt;

        
        &lt;hr&gt;
        &lt;address&gt;ApacheSling/2.2 (Day-Servlet-Engine/4.1.42, Java HotSpot(TM)
64-Bit Server VM 1.6.0_26, Windows 7 6.1 amd64)&lt;/address&gt;
    &lt;/body&gt;
&lt;/html&gt;
</pre>
</div></div>

<h3><a name="ChunkedFileUploadSupport-Abortchunkupload"></a>Abort chunk
upload</h3>
<p>Client will send delete" operation on upload id to  <a href="http://sling.apache.org/site/manipulating-content-the-slingpostservlet-servletspost.html"
class="external-link" rel="nofollow">SlingPostServlet</a> to abort chunk upload.<br/>
[request]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Abort chunk upload request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
POST /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08 HTTP/1.1
Authorization: Basic YWRtaW46YWRtaW4=
Content-Length: 207
Content-Type: multipart/form-data; boundary=tBnruF3YNdCs-besNxXn8zWO6bN1GL_
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Host: localhost:4502

--tBnruF3YNdCs-besNxXn8zWO6bN1GL_
Content-Disposition: form-data; name=<span class="code-quote">":operation"</span>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

delete
--tBnruF3YNdCs-besNxXn8zWO6bN1GL_--

</pre>
</div></div>
<p>[response]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Abort chunk upload response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
HTTP/1.1 200 OK
Connection: Keep-Alive
Server: Day-Servlet-Engine/4.1.42
Content-Type: text/html;charset=UTF-8
Date: Thu, 28 Feb 2013 14:14:25 GMT
Transfer-Encoding: chunked

&lt;html&gt;
&lt;head&gt;
    &lt;title&gt;Content modified /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08&lt;/title&gt;
&lt;/head&gt;
    &lt;body&gt;
    &lt;h1&gt;Content modified /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08&lt;/h1&gt;
    &lt;table&gt;
        &lt;tbody&gt;
            &lt;tr&gt;
                &lt;td&gt;Status&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Status"</span>&gt;200&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Message&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Message"</span>&gt;OK&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Location&lt;/td&gt;
                &lt;td&gt;&lt;img src=<span class="code-quote">"/libs/cq/linkchecker/resources/linkcheck_o.gif"</span>
alt=<span class="code-quote">"invalid link: /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>
title=<span class="code-quote">"invalid link: /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>
border=<span class="code-quote">"0"</span>&gt;/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08&lt;img
src=<span class="code-quote">"/libs/cq/linkchecker/resources/linkcheck_c.gif"</span>
border=<span class="code-quote">"0"</span>&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Parent Location&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"/tmp/sling/chunks"</span>
id=<span class="code-quote">"ParentLocation"</span>&gt;/tmp/sling/chunks&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Path&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Path"</span>&gt;/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Referer&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"" id="</span>Referer"&gt;&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;ChangeLog&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"ChangeLog"</span>&gt;&amp;lt;pre&amp;gt;deleted(<span
class="code-quote">"/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>);&amp;lt;br/&amp;gt;&amp;lt;/pre&amp;gt;&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
        &lt;/tbody&gt;
    &lt;/table&gt;
    &lt;p&gt;&lt;a href=""&gt;Go Back&lt;/a&gt;&lt;/p&gt;
    &lt;p&gt;&lt;img src=<span class="code-quote">"/libs/cq/linkchecker/resources/linkcheck_o.gif"</span>
alt=<span class="code-quote">"invalid link: /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>
title=<span class="code-quote">"invalid link: /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>
border=<span class="code-quote">"0"</span>&gt;Modified Resource&lt;img
src=<span class="code-quote">"/libs/cq/linkchecker/resources/linkcheck_c.gif"</span>
border=<span class="code-quote">"0"</span>&gt;&lt;/p&gt;
    &lt;p&gt;&lt;a href=<span class="code-quote">"/tmp/sling/chunks"</span>&gt;Parent
of Modified Resource&lt;/a&gt;&lt;/p&gt;
    &lt;/body&gt;
&lt;/html&gt;
</pre>
</div></div>

<h1><a name="ChunkedFileUploadSupport-Implementation"></a>Implementation</h1>

<h3><a name="ChunkedFileUploadSupport-ExtensiontoSlingPostServlet"></a>Extension
to <a href="http://sling.apache.org/site/manipulating-content-the-slingpostservlet-servletspost.html"
class="external-link" rel="nofollow">SlingPostServlet</a></h3>
<p>Chunk upload is supported in "modify"operation of <a href="http://sling.apache.org/site/manipulating-content-the-slingpostservlet-servletspost.html"
class="external-link" rel="nofollow">SlingPostServlet</a> ( ie. In default operation)
. No new operation introduced for it</p>
<h3><a name="ChunkedFileUploadSupport-ChunksStorage"></a>Chunks Storage</h3>
<p>Chunks are saved in temporary location identified by upload id in the following node
structure. <br/>
[request]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Get chunks information</b></div><div
class="codeContent panelContent">
<pre class="code-java">
GET /tmp/sling/chunks/f4bc84fbee4d4a608a87c1bf347ddded.infinity.json HTTP/1.1
Authorization: Basic YWRtaW46YWRtaW4=
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Host: localhost:4502
</pre>
</div></div>

<p>[response]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Get chunks information response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
HTTP/1.1 200 OK
Connection: Keep-Alive
Server: Day-Servlet-Engine/4.1.42
Content-Type: application/json;charset=UTF-8
Content-Length: 155
Date: Thu, 28 Feb 2013 15:09:53 GMT
{<span class="code-quote">"chunksUploaded"</span>:5,<span class="code-quote">"jcr:createdBy"</span>:<span
class="code-quote">"admin"</span>,<span class="code-quote">"bytesUploaded"</span>:1600,<span
class="code-quote">"jcr:created"</span>:<span class="code-quote">"Thu Feb 28
2013 20:39:53 GMT+0530"</span>,<span class="code-quote">"jcr:primaryType"</span>:<span
class="code-quote">"sling:OrderedFolder"</span>,<span class="code-quote">"1"</span>:{<span
class="code-quote">"size"</span>:200,<span class="code-quote">":jcr:data"</span>:200,<span
class="code-quote">"jcr:primaryType"</span>:<span class="code-quote">"nt:unstructured"</span>},<span
class="code-quote">"2"</span>:{<span class="code-quote">"size"</span>:300,<span
class="code-quote">":jcr:data"</span>:300,<span class="code-quote">"jcr:primaryType"</span>:<span
class="code-quote">"nt:unstructured"</span>},<span class="code-quote">"3"</span>:{<span
class="code-quote">"size"</span>:300,<span class="code-quote">":jcr:data"</span>:300,<span
class="code-quote">"jcr:primaryType"</span>:<span class="code-quote">"nt:unstructured"</span>},<span
class="code-quote">"4"</span>:{<span class="code-quote">"size"</span>:400,<span
class="code-quote">":jcr:data"</span>:400,<span class="code-quote">"jcr:primaryType"</span>:<span
class="code-quote">"nt:unstructured"</span>},<span class="code-quote">"5"</span>:{<span
class="code-quote">"size"</span>:400,<span class="code-quote">":jcr:data"</span>:400,<span
class="code-quote">"jcr:primaryType"</span>:<span class="code-quote">"nt:unstructured"</span>}}
</pre>
</div></div>
<h3><a name="ChunkedFileUploadSupport-Chunkprocessing"></a>Chunk processing</h3>
<ul>
	<li>First/Intermediate chunks: Chunk is saved in temporary location in jcr. Ignores
all upload semantics (@TypeHint, etc) and request parameters.</li>
	<li>Last chunk: Stitches all chunks. Process all upload semantics and request parameters
and creates jcr node structure at the right location.</li>
</ul>



    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href="https://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
        </div>
        <a href="https://cwiki.apache.org/confluence/display/SLING/Chunked+File+Upload+Support">View
Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=30749779&revisedVersion=34&originalVersion=33">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/SLING/Chunked+File+Upload+Support?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

Mime
View raw message