Mailing-List: contact dev-help@chemistry.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@chemistry.apache.org
Received-SPF: pass (athena.apache.org: domain of peter.schmidt@alfresco.com
 designates 207.126.144.115 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAPz_PHF30vQYfrNwZj_9LK05kpnGxk+YohEuvOe0e_+1E9-+VA@mail.gmail.com>
References: 
 <CAFzZK1bFMnPsbNWxWWo1VE3kdkUciwXmyF548Wt7RkA3=qix8w@mail.gmail.com>
	<CAPz_PHHxXAfDi_M_bgjfyFFUaoymsdnEKv8q9MK3bvwaBGajog@mail.gmail.com>
	<CAFzZK1YYip22y++6QZWtH-kRaC20e7DcS5p+pF6wAfv1-3n1QQ@mail.gmail.com>
	<CAPz_PHF30vQYfrNwZj_9LK05kpnGxk+YohEuvOe0e_+1E9-+VA@mail.gmail.com>
Date: Thu, 28 Feb 2013 10:38:38 +0000
Message-ID: 
 <CAFzZK1aeByoMCAGaAyKyOa4d+HKgMvLoJYpsvS2uBghs8aTiWg@mail.gmail.com>
Subject: Re: Objective-CMIS: base64 input streaming
From: Peter Schmidt <peter.schmidt@alfresco.com>
To: dev@chemistry.apache.org
Content-Type: multipart/alternative; boundary=bcaec555504056f6b804d6c67e98

--bcaec555504056f6b804d6c67e98
Content-Type: text/plain; charset=ISO-8859-1

Hi Joram
actually the "byte jiggling" refers only to the fact that the "inner" read
statement (i.e. from the original source) has to be a multiple of 3 until
the stream end is reached. I.e. the buffer size for the "inner" read and
the actual read are different.

It then simply boils down to storing the remaining until the next read
request from the HTTP connection.

I tested it with different byte sizes, and in fact our unit tests in
AlfrescoSDK as well as CMIS lib upload files of various sizes.

Regards
Peter


On 28 February 2013 09:53, Joram Barrez <joram.barrez@gmail.com> wrote:

> Yes, it does make sense. And I do agree that a stream implementation is
> better than the tmp file approach.
>
> But te 'byte jiggling' makes me kinda nervous. There's plenty that can go
> wrong once one decide to jiggle bytes ;-). So maybe add some more tests
> with different types of files (very small, very large, etc...) to the test
> suite so we have a bit more confidence in the implementation.
>
>
> On Thu, Feb 28, 2013 at 10:35 AM, Peter Schmidt
> <peter.schmidt@alfresco.com>wrote:
>
> > Hi Joram
> > indeed it took a bit of byte jiggling.
> >
> > The trick is to read an integer multiple of 3 from the non-encoded source
> > input stream. That way you guarantee that the base64 encoder won't pad
> the
> > encoded bytes with = at the end.
> > The exception, of course, is when you reach the end of the source stream,
> > where = padding is expected.
> >
> > The second trick is to write out as many bytes as are requested in the
> > read:maxLength statement. This means, filling the buffer with XML and
> > base64 encoded data - and storing the remainder in a tmp buffer until the
> > next read.
> >
> > We also need to ensure that the base64 input stream returns "has bytes
> > available" even if the source input stream is exhausted.
> >
> > I tested it in the XCode profiler, and the memory foot print seems to be
> > comparable with what we had before.
> >
> > Hope that makes sense?
> >
> > Regards
> > Peter
> >
> >
> >
> > On 28 February 2013 09:17, Joram Barrez <joram.barrez@gmail.com> wrote:
> >
> > > > when a read request is being sent to the new input stream class, the
> > XML
> > > > encapsulation and base64 encoding is done on the fly
> > >
> > > I remember I looked around for a streaming base64 implementation, but
> > > couldn't find something ready to use in ios straight away.
> > >
> > > Are you using some library or are you keeping some characters in memory
> > > until you have enough to do the encoding (like described eg in
> > >
> > >
> >
> http://stackoverflow.com/questions/7920780/is-it-possible-to-base64-encode-a-file-in-chunks
> > > )?
> > >
> > >
> > >
> > >
> > > On Thu, Feb 28, 2013 at 10:03 AM, Peter Schmidt
> > > <peter.schmidt@alfresco.com>wrote:
> > >
> > > > Hi all
> > > > just a brief heads-up:
> > > > I am going to submit a change in the way content is being uploaded
> from
> > > the
> > > > device to the server.
> > > > How it works at present:
> > > > * a temporary file is created containing the CMIS XML and base64
> > encoded
> > > > data.
> > > > * a NSInputStream pointing to the tmp file is created
> > > > * the HTTP request BodyStream property is being set to this tmp input
> > > > stream
> > > >
> > > > The new code will do the following:
> > > > * a new class CMISBase64InputStream is being provided, which inherits
> > > from
> > > > NSInputStream. It is being initialised with the original source input
> > > > stream (raw data).
> > > > * the HTTP request BodyStream property is being set to the
> > > > CMISBase64InputStream object.
> > > > * when a read request is being sent to the new input stream class,
> the
> > > XML
> > > > encapsulation and base64 encoding is done on the fly
> > > >
> > > > I tested it in both unit tests and a sample app, and it seems to
> work.
> > > > Unless there are objections, I will submit the new code by end of
> play
> > > > tomorrow, Fri 1 March
> > > >
> > > > --
> > > > Kind regards
> > > > Peter
> > > >
> > > > -----------
> > > > *Peter Schmidt*
> > > > *Alfresco Software Ltd.*
> > > > *UK: 07748 185496*
> > > > *Int.: 0044 7748 185496*
> > > > *Skype: pweschmidt*
> > > >
> > >
> >
> >
> >
> > --
> > Kind regards
> > Peter
> >
> > -----------
> > *Peter Schmidt*
> > *Alfresco Software Ltd.*
> > *UK: 07748 185496*
> > *Int.: 0044 7748 185496*
> > *Skype: pweschmidt*
> >
>


-- 
Kind regards
Peter

-----------
*Peter Schmidt*
*Alfresco Software Ltd.*
*UK: 07748 185496*
*Int.: 0044 7748 185496*
*Skype: pweschmidt*

--bcaec555504056f6b804d6c67e98--