incubator-sling-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Sling > Chunked File Upload Support
Date Mon, 06 May 2013 16:12:00 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/2042/9/1/_/styles/combined.css?spaceKey=SLING&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/SLING/Chunked+File+Upload+Support">Chunked
File Upload Support</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~shgupta">Shashank
Gupta</a>
    </h4>
        <br/>
                         <h4>Changes (2)</h4>
                                 
    
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
    
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >sling:chunksLength property indicate
cumulative length of all chunks stored in sling. Sling client should resume upload from sling:chunksLength
property&#39;s value offset <br>{code:title= Query interrupted chunk upload response}
<br></td></tr>
            <tr><td class="diff-deleted-lines" style="color:#999;background-color:#fdd;text-decoration:line-through;">{&quot;jcr:createdBy&quot;:&quot;admin&quot;,&quot;jcr:created&quot;:&quot;Mon
May 06 2013 21:31:22 GMT+0530&quot;,&quot;jcr:primaryType&quot;:&quot;nt:file&quot;,&quot;jcr:content&quot;:{&quot;jcr:lastModifiedBy&quot;:&quot;admin&quot;,&quot;jcr:uuid&quot;:&quot;845e9cee-f963-4f72-b115-fa021859c809&quot;,&quot;:jcr:data&quot;:0,&quot;jcr:mixinTypes&quot;:[&quot;sling:chunkMixin&quot;],&quot;sling:chunksLength&quot;:200,&quot;jcr:mimeType&quot;:&quot;application/pdf&quot;,&quot;jcr:lastModified&quot;:&quot;Mon
May 06 2013 21:31:22 GMT+0530&quot;,&quot;jcr:primaryType&quot;:&quot;nt:resource&quot;,&quot;sling:fileLength&quot;:1700,&quot;chunk_0_199&quot;:{&quot;jcr:createdBy&quot;:&quot;admin&quot;,&quot;:jcr:data&quot;:200,&quot;sling:offset&quot;:0,&quot;jcr:created&quot;:&quot;Mon
May 06 2013 21:31:22 GMT+0530&quot;,&quot;jcr:primaryType&quot;:&quot;sling:chunkNode&quot;}}
<br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">{&quot;jcr:createdBy&quot;:&quot;admin&quot;,&quot;jcr:created&quot;:&quot;Mon
May 06 2013 21:31:22 GMT+0530&quot;,&quot;jcr:primaryType&quot;:&quot;nt:file&quot;,&quot;jcr:content&quot;:{&quot;jcr:lastModifiedBy&quot;:&quot;admin&quot;,&quot;jcr:uuid&quot;:&quot;845e9cee-f963-4f72-b115-fa021859c809&quot;,
<br>&quot;:jcr:data&quot;:0,&quot;jcr:mixinTypes&quot;:[&quot;sling:chunkMixin&quot;],&quot;sling:chunksLength&quot;:200,&quot;jcr:mimeType&quot;:&quot;application/pdf&quot;,&quot;jcr:lastModified&quot;:&quot;Mon
May 06 2013 21:31:22 GMT+0530&quot;, <br>&quot;jcr:primaryType&quot;:&quot;nt:resource&quot;,&quot;sling:fileLength&quot;:1700,&quot;chunk_0_199&quot;:{&quot;jcr:createdBy&quot;:&quot;admin&quot;,&quot;:jcr:data&quot;:200,&quot;sling:offset&quot;:0,&quot;jcr:created&quot;:&quot;Mon
May 06 2013 21:31:22 GMT+0530&quot;,&quot;jcr:primaryType&quot;:&quot;sling:chunkNode&quot;}}
<br></td></tr>
            <tr><td class="diff-unchanged" >{code} <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
    
            </table>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <div>
<ul>
    <li><span class='TOCOutline'>1</span> <a href='#ChunkedFileUploadSupport-ChunkedFileUpload'>Chunked
File Upload</a></li>
    <li><span class='TOCOutline'>2</span> <a href='#ChunkedFileUploadSupport-UseCases%3A'>Use
Cases:</a></li>
    <li><span class='TOCOutline'>3</span> <a href='#ChunkedFileUploadSupport-Approach'>Approach</a></li>
    <li><span class='TOCOutline'>4</span> <a href='#ChunkedFileUploadSupport-ContentModel'>Content
Model</a></li>
    <li><span class='TOCOutline'>5</span> <a href='#ChunkedFileUploadSupport-ProtocolSpecification'>Protocol
Specification</a></li>
<ul>
    <li><span class='TOCOutline'>5.1</span> <a href='#ChunkedFileUploadSupport-UploadchunkusingPOST'>Upload
chunk using POST</a></li>
    <li><span class='TOCOutline'>5.2</span> <a href='#ChunkedFileUploadSupport-QuerySlingabouttheinterruptedchunkuploadstatus'>Query
Sling about the interrupted chunk upload status</a></li>
    <li><span class='TOCOutline'>5.3</span> <a href='#ChunkedFileUploadSupport-Abortchunkupload'>Abort
chunk upload</a></li>
</ul>
    <li><span class='TOCOutline'>6</span> <a href='#ChunkedFileUploadSupport-Implementation'>Implementation</a></li>
<ul>
    <li><span class='TOCOutline'>6.1</span> <a href='#ChunkedFileUploadSupport-ExtensiontoSlingPostServlet'>Extension
to SlingPostServlet</a></li>
    <li><span class='TOCOutline'>6.2</span> <a href='#ChunkedFileUploadSupport-ChunksStorage'>Chunks
Storage</a></li>
    <li><span class='TOCOutline'>6.3</span> <a href='#ChunkedFileUploadSupport-Chunkprocessing'>Chunk
processing</a></li>
</ul>
</ul></div>

<h1><a name="ChunkedFileUploadSupport-ChunkedFileUpload"></a>Chunked File
Upload</h1>

<p>Status: DRAFT<br/>
Created: 20. January 2013<br/>
Author: shgupta<br/>
JIRA: <a href="https://issues.apache.org/jira/browse/SLING-2707" class="external-link"
rel="nofollow">SLING-2707</a><br/>
References: - <a href="http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html"
class="external-link" rel="nofollow">http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html</a><br/>
Updated: -</p>

<h1><a name="ChunkedFileUploadSupport-UseCases%3A"></a>Use Cases:</h1>
<p>1. Large file upload - With high speed internet connections, advent of cloud and
HD going mainstream, Sling support large files (&gt; 2GB) upload.<br/>
2. Fault tolerant uploads - Sling provides capability to resume upload from failure point.
It doesn't require client to restart the complete upload process. </p>

<h1><a name="ChunkedFileUploadSupport-Approach"></a>Approach</h1>
<p>Sling provides an extension to <a href="http://sling.apache.org/site/manipulating-content-the-slingpostservlet-servletspost.html"
class="external-link" rel="nofollow">SlingPostServlet</a> which accepts file chunks
in accordance with a specified protocol. Sling client slices the file in chunks, and upload
the chunks in serial manner to server. Each chunk has "Offset" attribute which identify chunk
position in complete file. <a href="http://sling.apache.org/site/manipulating-content-the-slingpostservlet-servletspost.html"
class="external-link" rel="nofollow">SlingPostServlet</a>upon receiving the last
chunk, stitches all chunks into a single file and save them to the final destination. </p>

<p>In case of upload failures, sling provides support to query the last chunk uploaded
till failure point. Client resumes chunk upload from last failure point. </p>

<h1><a name="ChunkedFileUploadSupport-ContentModel"></a>Content Model</h1>
<p>Chunks are stored within actual path in sling:chunkMixin mixin node type. The content
model to store chunk is defined as follows:</p>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Content model to store chunk</b></div><div
class="codeContent panelContent">
<pre class="code-java">
<span class="code-comment">// node type to store chunk
</span><span class="code-comment">// offset: offset of chunk in file
</span><span class="code-comment">// jcr:data: binary of chunk
</span>[sling:chunkNode] &gt; nt:hierarchyNode
  primaryitem jcr:data
  - sling:offset  (<span class="code-object">long</span>) mandatory
  - jcr:data (binary) mandatory
 
 <span class="code-comment">//-----------------------------------------------------------------------------

</span> <span class="code-comment">// Mixin node type to identify that a node
has chunks
</span> <span class="code-comment">// sling:fileLength : length of complete file
</span> <span class="code-comment">// sling:chunksLength: cumulative length of
all uploaded chunks
</span>[sling:chunkMixin]
  mixin
  - sling:fileLength (<span class="code-object">long</span>)
  - sling:chunksLength (<span class="code-object">long</span>)
  + * (sling:chunkNode) multiple
</pre>
</div></div>

<p>The typical nt:file node under chunked upload would look like </p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Typical nt:file nod under chunked upload</b></div><div
class="codeContent panelContent">
<pre class="code-java">
/content/dam/folder/catalog.pdf [nt:file]
                                + jcr:content [nt:resource] [sling:chunkMixin]
                                    - jcr:data = empty until completed
                                    - sling:fileLength = 982145 <span class="code-comment">//
(filename@Length from client)
</span>									- sling:chunksLength = 30000 <span class="code-comment">//cumulative
length of all uploaded chunks
</span>                                    + chunk_0-9999 [sling:chunkNode]
                                            - sling:offset =0
                                            - jcr:data [binary data]
                                    + chunk_10000-19999 [sling:chunkNode]
                                    + ....
</pre>
</div></div>

<h1><a name="ChunkedFileUploadSupport-ProtocolSpecification"></a>Protocol
Specification </h1>


<h3><a name="ChunkedFileUploadSupport-UploadchunkusingPOST"></a>Upload chunk
using POST </h3>

<h4><a name="ChunkedFileUploadSupport-Chunkuploadrequest"></a>Chunk upload
request</h4>
<p>Client uses POST method to parent path to upload binary chunk of file. Sling client
passes $filename@Offset and $filename@Length as multipart request parameters. "Offset" indicate
chunk's data offset in complete file. "Length" is optional parameter and indicates length
of complete file. If "Length" parameter is known, Sling automatically calculates if request
is last chunk request and according stitches all chunks and store it into final destination.
<br/>
[request]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>First/Intermediate chunk upload request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
POST /content/dam/folder HTTP/1.1
Authorization: Basic YWRtaW46YWRtaW4=
Transfer-Encoding: chunked
Content-Type: multipart/form-data; boundary=CbZDcL_DxJIVQqSG1WkYaIoLWqT3FGYCVe
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Host: localhost:4502

--CbZDcL_DxJIVQqSG1WkYaIoLWqT3FGYCVe
Content-Disposition: form-data; name=<span class="code-quote">"catalog.pdf@Length"</span>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

1000
--CbZDcL_DxJIVQqSG1WkYaIoLWqT3FGYCVe
Content-Disposition: form-data; name=<span class="code-quote">"catalog.pdf@Offset"</span>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

400
--CbZDcL_DxJIVQqSG1WkYaIoLWqT3FGYCVe
Content-Disposition: form-data; name=<span class="code-quote">"catalog.pdf"</span>;
filename=<span class="code-quote">"catalog.pdf"</span>
Content-Type: application/pdf
Content-Transfer-Encoding: binary
$binary-data
--CbZDcL_DxJIVQqSG1WkYaIoLWqT3FGYCVe--
</pre>
</div></div>

<p>[response]<br/>
The response shows that chunk is stored within the actual path in sling:chunkMixin node type</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Chunk upload response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
HTTP/1.1 200 OK
Connection: Keep-Alive
Server: Day-Servlet-Engine/4.1.42
Content-Type: text/html;charset=UTF-8
Date: Mon, 06 May 2013 14:42:22 GMT
Transfer-Encoding: chunked

&lt;html&gt;
&lt;head&gt;
    &lt;title&gt;Content modified /content/dam/folder&lt;/title&gt;
&lt;/head&gt;
    &lt;body&gt;
    &lt;h1&gt;Content modified /content/dam/folder&lt;/h1&gt;
    &lt;table&gt;
        &lt;tbody&gt;
            &lt;tr&gt;
                &lt;td&gt;Status&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Status"</span>&gt;200&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Message&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Message"</span>&gt;OK&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Location&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"/content/dam/folder"</span>
id=<span class="code-quote">"Location"</span>&gt;/content/dam/folder&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Parent Location&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"/"</span>
id=<span class="code-quote">"ParentLocation"</span>&gt;/&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Path&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Path"</span>&gt;/content/dam/folder&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Referer&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"" id="</span>Referer"&gt;&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;ChangeLog&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"ChangeLog"</span>&gt;&amp;lt;pre&amp;gt;modified(<span
class="code-quote">"/content/dam/folder/catalog.pdf/jcr:content/jcr:lastModified"</span>);&amp;lt;br/&amp;gt;modified(<span
class="code-quote">"/content/dam/folder/catalog.pdf/jcr:content/jcr:mimeType"</span>);
&amp;lt;br/&amp;gt;created(<span class="code-quote">"/content/dam/folder/catalog.pdf/jcr:content/chunk_400_799"</span>);&amp;lt;br/&amp;gt;modified(<span
class="code-quote">"/content/dam/folder/catalog.pdf/jcr:content/chunk_400_799/jcr:data"</span>);&amp;lt;br/&amp;gt;
modified(<span class="code-quote">"/content/dam/folder/catalog.pdf/jcr:content/chunk_400_799/sling:offset"</span>);&amp;lt;br/&amp;gt;modified(<span
class="code-quote">"/content/dam/folder/catalog.pdf/jcr:content/sling:chunksLength"</span>);&amp;lt;br/&amp;gt;&amp;lt;/pre&amp;gt;&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
        &lt;/tbody&gt;
    &lt;/table&gt;
    &lt;p&gt;&lt;a href=""&gt;Go Back&lt;/a&gt;&lt;/p&gt;
    &lt;p&gt;&lt;a href=<span class="code-quote">"/content/dam/folder"</span>&gt;Modified
Resource&lt;/a&gt;&lt;/p&gt;
    &lt;p&gt;&lt;a href=<span class="code-quote">"/"</span>&gt;Parent
of Modified Resource&lt;/a&gt;&lt;/p&gt;
    &lt;/body&gt;
&lt;/html&gt;
</pre>
</div></div>

<h4><a name="ChunkedFileUploadSupport-ChunkUploadinstreamingusecase"></a>Chunk
Upload in streaming use case</h4>
<p>[request]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Last chunk upload request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
POST /content/dam/folder HTTP/1.1
Authorization: Basic YWRtaW46YWRtaW4=
Transfer-Encoding: chunked
Content-Type: multipart/form-data; boundary=lMaKIb2KPscWvPV8B0fULKkKayVtcxugD8Lt
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Host: localhost:4502

--lMaKIb2KPscWvPV8B0fULKkKayVtcxugD8Lt
Content-Disposition: form-data; name=<span class="code-quote">"catalog.pdf@Completed"</span>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

<span class="code-keyword">true</span>
--lMaKIb2KPscWvPV8B0fULKkKayVtcxugD8Lt
Content-Disposition: form-data; name=<span class="code-quote">"catalog.pdf@Offset"</span>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

400
--lMaKIb2KPscWvPV8B0fULKkKayVtcxugD8Lt
Content-Disposition: form-data; name=<span class="code-quote">"catalog.pdf"</span>;
filename=<span class="code-quote">"catalog.pdf"</span>
Content-Type: application/pdf
Content-Transfer-Encoding: binary
$binary_data

--lMaKIb2KPscWvPV8B0fULKkKayVtcxugD8Lt--
</pre>
</div></div>
<p>[response]<br/>
The response shows that merge chunks uploaded at final destination and deletion of chunk upload
metadata.</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Last chunk upload response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
HTTP/1.1 200 OK
Connection: Keep-Alive
Server: Day-Servlet-Engine/4.1.42
Content-Type: text/html;charset=UTF-8
Date: Mon, 06 May 2013 15:52:16 GMT
Transfer-Encoding: chunked

&lt;html&gt;
&lt;head&gt;
    &lt;title&gt;Content modified /content/dam/folder&lt;/title&gt;
&lt;/head&gt;
    &lt;body&gt;
    &lt;h1&gt;Content modified /content/dam/folder&lt;/h1&gt;
    &lt;table&gt;
        &lt;tbody&gt;
            &lt;tr&gt;
                &lt;td&gt;Status&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Status"</span>&gt;200&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Message&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Message"</span>&gt;OK&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Location&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"/content/dam/folder"</span>
id=<span class="code-quote">"Location"</span>&gt;/content/dam/folder&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Parent Location&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"/"</span>
id=<span class="code-quote">"ParentLocation"</span>&gt;/&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Path&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Path"</span>&gt;/content/dam/folder&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Referer&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"" id="</span>Referer"&gt;&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;ChangeLog&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"ChangeLog"</span>&gt;&amp;lt;pre&amp;gt;modified(<span
class="code-quote">"/content/dam/folder/catalog.pdf/jcr:content/jcr:lastModified"</span>);&amp;lt;br/&amp;gt;modified(<span
class="code-quote">"/content/dam/folder/catalog.pdf/jcr:content/jcr:mimeType"</span>);
&amp;lt;br/&amp;gt;modified(<span class="code-quote">"/content/dam/folder/catalog.pdf/jcr:content/jcr:data"</span>);&amp;lt;br/&amp;gt;deleted(<span
class="code-quote">"/content/dam/folder/catalog.pdf/jcr:content/chunk_0_199"</span>);
&amp;lt;br/&amp;gt;deleted(<span class="code-quote">"/content/dam/folder/catalog.pdf/jcr:content/chunk_200_399"</span>);&amp;lt;br/&amp;gt;deleted(<span
class="code-quote">"/content/dam/folder/catalog.pdf/jcr:content/sling:chunksLength"</span>);
&amp;lt;br/&amp;gt;&amp;lt;/pre&amp;gt;&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
        &lt;/tbody&gt;
    &lt;/table&gt;
    &lt;p&gt;&lt;a href=""&gt;Go Back&lt;/a&gt;&lt;/p&gt;
    &lt;p&gt;&lt;a href=<span class="code-quote">"/content/dam/folder"</span>&gt;Modified
Resource&lt;/a&gt;&lt;/p&gt;
    &lt;p&gt;&lt;a href=<span class="code-quote">"/"</span>&gt;Parent
of Modified Resource&lt;/a&gt;&lt;/p&gt;
    &lt;/body&gt;
&lt;/html&gt;
</pre>
</div></div>



<h3><a name="ChunkedFileUploadSupport-QuerySlingabouttheinterruptedchunkuploadstatus"></a>Query
Sling about the interrupted chunk upload status</h3>
<p>Client sent get request on upload to retrieve chunk upload status.<br/>
[request]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Query interrupted chunk upload request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
GET <span class="code-comment">//content/dam/folder/catalog.pdf.3.json HTTP/1.1
</span>Authorization: Basic YWRtaW46YWRtaW4=
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Host: localhost:4502
</pre>
</div></div>
<p>[response]<br/>
sling:chunksLength property indicate cumulative length of all chunks stored in sling. Sling
client should resume upload from sling:chunksLength property's value offset</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Query interrupted chunk upload response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
{<span class="code-quote">"jcr:createdBy"</span>:<span class="code-quote">"admin"</span>,<span
class="code-quote">"jcr:created"</span>:<span class="code-quote">"Mon May 06
2013 21:31:22 GMT+0530"</span>,<span class="code-quote">"jcr:primaryType"</span>:<span
class="code-quote">"nt:file"</span>,<span class="code-quote">"jcr:content"</span>:{<span
class="code-quote">"jcr:lastModifiedBy"</span>:<span class="code-quote">"admin"</span>,<span
class="code-quote">"jcr:uuid"</span>:<span class="code-quote">"845e9cee-f963-4f72-b115-fa021859c809"</span>,
<span class="code-quote">":jcr:data"</span>:0,<span class="code-quote">"jcr:mixinTypes"</span>:[<span
class="code-quote">"sling:chunkMixin"</span>],<span class="code-quote">"sling:chunksLength"</span>:200,<span
class="code-quote">"jcr:mimeType"</span>:<span class="code-quote">"application/pdf"</span>,<span
class="code-quote">"jcr:lastModified"</span>:<span class="code-quote">"Mon
May 06 2013 21:31:22 GMT+0530"</span>,
<span class="code-quote">"jcr:primaryType"</span>:<span class="code-quote">"nt:resource"</span>,<span
class="code-quote">"sling:fileLength"</span>:1700,<span class="code-quote">"chunk_0_199"</span>:{<span
class="code-quote">"jcr:createdBy"</span>:<span class="code-quote">"admin"</span>,<span
class="code-quote">":jcr:data"</span>:200,<span class="code-quote">"sling:offset"</span>:0,<span
class="code-quote">"jcr:created"</span>:<span class="code-quote">"Mon May 06
2013 21:31:22 GMT+0530"</span>,<span class="code-quote">"jcr:primaryType"</span>:<span
class="code-quote">"sling:chunkNode"</span>}}
</pre>
</div></div>

<h4><a name="ChunkedFileUploadSupport-Ifnobroken%2Fdiscontinueduploadslingreturns404notfoundresponse."></a>If
no broken/discontinued upload sling returns 404 not found response. </h4>
<p>[response]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Chunk upload not found response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
HTTP/1.1 404 Not Found
Connection: Close
Server: Day-Servlet-Engine/4.1.42
Content-Type: text/html;charset=utf-8
Date: Thu, 28 Feb 2013 14:14:25 GMT
Transfer-Encoding: chunked

&lt;!DOCTYPE HTML PUBLIC <span class="code-quote">"-<span class="code-comment">//IETF//DTD
HTML 2.0//EN"</span>&gt;
</span>&lt;html&gt;
    &lt;head&gt;&lt;title&gt;404 No resource found&lt;/title&gt;&lt;/head&gt;
    &lt;body&gt;
        &lt;h1&gt;No resource found&lt;/h1&gt;
        &lt;p&gt;Cannot serve request to /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08
in /libs/sling/servlet/errorhandler/404.jsp&lt;/p&gt;
        
        &lt;h3&gt;Request Progress:&lt;/h3&gt;
&lt;pre&gt;
      0 (2013-02-28 19:44:25) TIMER_START{Request Processing}
      0 (2013-02-28 19:44:25) COMMENT timer_end format is {&amp;lt;elapsed msec&amp;gt;,&amp;lt;timer
name&amp;gt;} &amp;lt;optional message&amp;gt;
      0 (2013-02-28 19:44:25) LOG Method=GET, PathInfo=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08
      0 (2013-02-28 19:44:25) TIMER_START{ResourceResolution}
      0 (2013-02-28 19:44:25) TIMER_END{0,ResourceResolution} URI=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08
resolves to Resource=, type=sling:nonexisting, path=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08,
resource=[NonExistingResource, path=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08]
      0 (2013-02-28 19:44:25) LOG Resource Path Info: SlingRequestPathInfo: path='/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08',
selectorString='<span class="code-keyword">null</span>', extension='<span class="code-keyword">null</span>',
suffix='<span class="code-keyword">null</span>'
      0 (2013-02-28 19:44:25) TIMER_START{ServletResolution}
      0 (2013-02-28 19:44:25) TIMER_START{resolveServlet(, type=sling:nonexisting, path=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08,
resource=[NonExistingResource, path=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08])}
      0 (2013-02-28 19:44:25) LOG {0}: no servlet found
      0 (2013-02-28 19:44:25) TIMER_END{0,resolveServlet(, type=sling:nonexisting, path=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08,
resource=[NonExistingResource, path=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08])}
Using servlet org.apache.sling.servlets.get.DefaultGetServlet
      0 (2013-02-28 19:44:25) TIMER_END{0,ServletResolution} URI=/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08
handled by Servlet=org.apache.sling.servlets.get.DefaultGetServlet
      0 (2013-02-28 19:44:25) LOG Applying Requestfilters
      0 (2013-02-28 19:44:25) LOG Calling filter: org.apache.sling.bgservlets.impl.BackgroundServletStarterFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: org.apache.sling.i18n.impl.I18NFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: org.apache.sling.rewriter.impl.RewriterFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.designimporter.CanvasPageDeleteRequestFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.core.impl.WCMRequestFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.adobe.granite.optout.impl.OptOutFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.theme.impl.ThemeResolverFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.foundation.forms.impl.FormsHandlingServlet
      0 (2013-02-28 19:44:25) LOG Calling filter: org.apache.sling.engine.impl.debug.RequestProgressTrackerLogFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.mobile.core.impl.redirect.RedirectFilter
      0 (2013-02-28 19:44:25) LOG RedirectFilter did not redirect (request extension does
not match)
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.core.impl.warp.TimeWarpFilter
      0 (2013-02-28 19:44:25) LOG Applying Componentfilters
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.core.impl.WCMComponentFilter
      0 (2013-02-28 19:44:25) LOG Calling filter: com.day.cq.wcm.core.impl.WCMDebugFilter
      0 (2013-02-28 19:44:25) TIMER_START{org.apache.sling.servlets.get.DefaultGetServlet#0}
      0 (2013-02-28 19:44:25) TIMER_END{0,org.apache.sling.servlets.get.DefaultGetServlet#0}
      0 (2013-02-28 19:44:25) LOG Applying Error filters
      0 (2013-02-28 19:44:25) LOG Calling filter: org.apache.sling.rewriter.impl.RewriterFilter
      0 (2013-02-28 19:44:25) TIMER_START{handleError:status=404}
      0 (2013-02-28 19:44:25) TIMER_END{0,handleError:status=404} Using handler /libs/sling/servlet/errorhandler/404.jsp
      0 (2013-02-28 19:44:25) LOG Found processor <span class="code-keyword">for</span>
post processing ProcessorConfiguration: {contentTypes=[text/html],order=-1, active=<span
class="code-keyword">true</span>, valid=<span class="code-keyword">true</span>,
processErrorResponse=<span class="code-keyword">true</span>, pipeline=(generator=Config(type=htmlparser,
config={}), transformers=(Config(type=linkchecker, config={}), Config(type=mobile, config=org.apache.sling.jcr.resource.JcrPropertyMap@42e95cbf),
Config(type=mobiledebug, config=org.apache.sling.jcr.resource.JcrPropertyMap@66c821ea), Config(type=contentsync,
config=org.apache.sling.jcr.resource.JcrPropertyMap@5d2abe5d), serializer=Config(type=htmlwriter,
config={}))}
      0 (2013-02-28 19:44:25) TIMER_END{0,Request Processing} Dumping SlingRequestProgressTracker
Entries
&lt;/pre&gt;

        
        &lt;hr&gt;
        &lt;address&gt;ApacheSling/2.2 (Day-Servlet-Engine/4.1.42, Java HotSpot(TM)
64-Bit Server VM 1.6.0_26, Windows 7 6.1 amd64)&lt;/address&gt;
    &lt;/body&gt;
&lt;/html&gt;
</pre>
</div></div>

<h3><a name="ChunkedFileUploadSupport-Abortchunkupload"></a>Abort chunk
upload</h3>
<p>Client will send delete" operation on upload id to  <a href="http://sling.apache.org/site/manipulating-content-the-slingpostservlet-servletspost.html"
class="external-link" rel="nofollow">SlingPostServlet</a> to abort chunk upload.<br/>
[request]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Abort chunk upload request</b></div><div
class="codeContent panelContent">
<pre class="code-java">
POST /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08 HTTP/1.1
Authorization: Basic YWRtaW46YWRtaW4=
Content-Length: 207
Content-Type: multipart/form-data; boundary=tBnruF3YNdCs-besNxXn8zWO6bN1GL_
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Host: localhost:4502

--tBnruF3YNdCs-besNxXn8zWO6bN1GL_
Content-Disposition: form-data; name=<span class="code-quote">":operation"</span>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

delete
--tBnruF3YNdCs-besNxXn8zWO6bN1GL_--

</pre>
</div></div>
<p>[response]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Abort chunk upload response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
HTTP/1.1 200 OK
Connection: Keep-Alive
Server: Day-Servlet-Engine/4.1.42
Content-Type: text/html;charset=UTF-8
Date: Thu, 28 Feb 2013 14:14:25 GMT
Transfer-Encoding: chunked

&lt;html&gt;
&lt;head&gt;
    &lt;title&gt;Content modified /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08&lt;/title&gt;
&lt;/head&gt;
    &lt;body&gt;
    &lt;h1&gt;Content modified /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08&lt;/h1&gt;
    &lt;table&gt;
        &lt;tbody&gt;
            &lt;tr&gt;
                &lt;td&gt;Status&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Status"</span>&gt;200&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Message&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Message"</span>&gt;OK&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Location&lt;/td&gt;
                &lt;td&gt;&lt;img src=<span class="code-quote">"/libs/cq/linkchecker/resources/linkcheck_o.gif"</span>
alt=<span class="code-quote">"invalid link: /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>
title=<span class="code-quote">"invalid link: /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>
border=<span class="code-quote">"0"</span>&gt;/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08&lt;img
src=<span class="code-quote">"/libs/cq/linkchecker/resources/linkcheck_c.gif"</span>
border=<span class="code-quote">"0"</span>&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Parent Location&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"/tmp/sling/chunks"</span>
id=<span class="code-quote">"ParentLocation"</span>&gt;/tmp/sling/chunks&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Path&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"Path"</span>&gt;/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;Referer&lt;/td&gt;
                &lt;td&gt;&lt;a href=<span class="code-quote">"" id="</span>Referer"&gt;&lt;/a&gt;&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
                &lt;td&gt;ChangeLog&lt;/td&gt;
                &lt;td&gt;&lt;div id=<span class="code-quote">"ChangeLog"</span>&gt;&amp;lt;pre&amp;gt;deleted(<span
class="code-quote">"/tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>);&amp;lt;br/&amp;gt;&amp;lt;/pre&amp;gt;&lt;/div&gt;&lt;/td&gt;
            &lt;/tr&gt;
        &lt;/tbody&gt;
    &lt;/table&gt;
    &lt;p&gt;&lt;a href=""&gt;Go Back&lt;/a&gt;&lt;/p&gt;
    &lt;p&gt;&lt;img src=<span class="code-quote">"/libs/cq/linkchecker/resources/linkcheck_o.gif"</span>
alt=<span class="code-quote">"invalid link: /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>
title=<span class="code-quote">"invalid link: /tmp/sling/chunks/21d36856b76045da8719ac932f88fe08"</span>
border=<span class="code-quote">"0"</span>&gt;Modified Resource&lt;img
src=<span class="code-quote">"/libs/cq/linkchecker/resources/linkcheck_c.gif"</span>
border=<span class="code-quote">"0"</span>&gt;&lt;/p&gt;
    &lt;p&gt;&lt;a href=<span class="code-quote">"/tmp/sling/chunks"</span>&gt;Parent
of Modified Resource&lt;/a&gt;&lt;/p&gt;
    &lt;/body&gt;
&lt;/html&gt;
</pre>
</div></div>

<h1><a name="ChunkedFileUploadSupport-Implementation"></a>Implementation</h1>

<h3><a name="ChunkedFileUploadSupport-ExtensiontoSlingPostServlet"></a>Extension
to <a href="http://sling.apache.org/site/manipulating-content-the-slingpostservlet-servletspost.html"
class="external-link" rel="nofollow">SlingPostServlet</a></h3>
<p>Chunk upload is supported in "modify"operation of <a href="http://sling.apache.org/site/manipulating-content-the-slingpostservlet-servletspost.html"
class="external-link" rel="nofollow">SlingPostServlet</a> ( ie. In default operation)
. No new operation introduced for it</p>
<h3><a name="ChunkedFileUploadSupport-ChunksStorage"></a>Chunks Storage</h3>
<p>Chunks are saved in temporary location identified by upload id in the following node
structure. <br/>
[request]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Get chunks information</b></div><div
class="codeContent panelContent">
<pre class="code-java">
GET /tmp/sling/chunks/f4bc84fbee4d4a608a87c1bf347ddded.infinity.json HTTP/1.1
Authorization: Basic YWRtaW46YWRtaW4=
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Host: localhost:4502
</pre>
</div></div>

<p>[response]</p>
<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>Get chunks information response</b></div><div
class="codeContent panelContent">
<pre class="code-java">
HTTP/1.1 200 OK
Connection: Keep-Alive
Server: Day-Servlet-Engine/4.1.42
Content-Type: application/json;charset=UTF-8
Content-Length: 155
Date: Thu, 28 Feb 2013 15:09:53 GMT
{<span class="code-quote">"chunksUploaded"</span>:5,<span class="code-quote">"jcr:createdBy"</span>:<span
class="code-quote">"admin"</span>,<span class="code-quote">"bytesUploaded"</span>:1600,<span
class="code-quote">"jcr:created"</span>:<span class="code-quote">"Thu Feb 28
2013 20:39:53 GMT+0530"</span>,<span class="code-quote">"jcr:primaryType"</span>:<span
class="code-quote">"sling:OrderedFolder"</span>,<span class="code-quote">"1"</span>:{<span
class="code-quote">"size"</span>:200,<span class="code-quote">":jcr:data"</span>:200,<span
class="code-quote">"jcr:primaryType"</span>:<span class="code-quote">"nt:unstructured"</span>},<span
class="code-quote">"2"</span>:{<span class="code-quote">"size"</span>:300,<span
class="code-quote">":jcr:data"</span>:300,<span class="code-quote">"jcr:primaryType"</span>:<span
class="code-quote">"nt:unstructured"</span>},<span class="code-quote">"3"</span>:{<span
class="code-quote">"size"</span>:300,<span class="code-quote">":jcr:data"</span>:300,<span
class="code-quote">"jcr:primaryType"</span>:<span class="code-quote">"nt:unstructured"</span>},<span
class="code-quote">"4"</span>:{<span class="code-quote">"size"</span>:400,<span
class="code-quote">":jcr:data"</span>:400,<span class="code-quote">"jcr:primaryType"</span>:<span
class="code-quote">"nt:unstructured"</span>},<span class="code-quote">"5"</span>:{<span
class="code-quote">"size"</span>:400,<span class="code-quote">":jcr:data"</span>:400,<span
class="code-quote">"jcr:primaryType"</span>:<span class="code-quote">"nt:unstructured"</span>}}
</pre>
</div></div>
<h3><a name="ChunkedFileUploadSupport-Chunkprocessing"></a>Chunk processing</h3>
<ul>
	<li>First/Intermediate chunks: Chunk is saved in temporary location in jcr. Ignores
all upload semantics (@TypeHint, etc) and request parameters.</li>
	<li>Last chunk: Stitches all chunks. Process all upload semantics and request parameters
and creates jcr node structure at the right location.</li>
</ul>



    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href="https://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
        </div>
        <a href="https://cwiki.apache.org/confluence/display/SLING/Chunked+File+Upload+Support">View
Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=30749779&revisedVersion=41&originalVersion=40">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/SLING/Chunked+File+Upload+Support?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

Mime
View raw message