chemistry-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Inouye, Brian" <Brian.Ino...@xerox.com>
Subject RE: create service with a large content stream causes a "Limit exceeded!" exception
Date Tue, 01 Apr 2014 21:44:02 GMT
Hi Florian,

This will be rather difficult to reproduce.  The "Limit exceeded!" server exception occurs
when the file is being uploaded from a particular Mobile App going through WiFi to my server.
 However, the exception does not occur if the same file is uploaded from the CMIS Workbench
to my server.  I hope this isn't a timing-sensitive problem.  I am going to look at a packet
trace to see if I can see any differences between how the Mobile App is transferring the file
vs. the CMIS Workbench.

... Brian ...

-----Original Message-----
From: Florian Müller [mailto:fmui@apache.org] 
Sent: Monday, March 31, 2014 12:31 PM
To: Inouye, Brian
Cc: dev@chemistry.apache.org
Subject: Re: create service with a large content stream causes a "Limit exceeded!" exception

Hi Brian,

I tried a lot but I cannot reproduce it.
Is it possible to reproduce it with a smaller content and send me the request? I would like
to debug this.


- Florian


> Hi Florian,
> 
> No, we did not change anything in the OpenCMIS code.
> 
> AtomEntryParser uses XMLUtils to create the parser:
> 
> 	XMLStreamReader parser = XMLUtils.createParser(cappedStream);
> 
> XMLUtils explicitly sets isCoalescing to FALSE.
> 
>         XML_INPUT_FACTORY.setProperty(XMLInputFactory.IS_COALESCING, 
> Boolean.FALSE);
> 
> ... Brian ...
> 
> -----Original Message-----
> From: Florian Müller [mailto:fmui@apache.org]
> Sent: Friday, March 28, 2014 1:38 PM
> To: Inouye, Brian
> Cc: dev@chemistry.apache.org
> Subject: Re: create service with a large content stream causes a 
> "Limit exceeded!" exception
> 
> Hi Brian,
> 
> I've revalidated that uploading huge documents works as expected.
> 
> What you are seeing is only possible if the property "javax.xml.stream.isCoalescing"
at the XMLInputFactory has been set to TRUE somehow. Did you change anything in the OpenCMIS
code?
> 
> 
> - Florian
> 
> 
>> Hi Brian,
>>
>> The parser should process the the content of base64 tag in small chunks.
>> For some reasons it tries to read much more than that in your setup.
>> I'll look into it.
>>
>> - Florian
>>
>>
>>> Hi Florian,
>>>
>>> Here is the exception stack.  It looks like it's using the Woodstox 
>>> parser (I assume that's what com.ctc.wstx is).  The exception is 
>>> happening when AtomEntryParser is calling readBase64, so I think it 
>>> doesn't like the amount of the base64 encoded content stream.  The 
>>> rest of the XML, excluding the content stream, is very small.
>>>
>>> We don't have control over the Client so our options are limited.
>>>
>>> ... Brian ...
>>>
>>>
>>> org.apache.chemistry.opencmis.commons.exceptions.CmisInvalidArgumentException:
>>>
>>> Limit exceeded!
>>>     at
>>>
>>> org.apache.chemistry.opencmis.server.shared.CappedInputStream.checkL
>>> e
>>> ngth(CappedInputStream.java:71)
>>>
>>>     at
>>>
>>> org.apache.chemistry.opencmis.server.shared.CappedInputStream.read(C
>>> a
>>> ppedInputStream.java:107)
>>>
>>>     at com.ctc.wstx.io.BaseReader.readBytes(BaseReader.java:155)
>>>     at com.ctc.wstx.io.UTF8Reader.loadMore(UTF8Reader.java:368)
>>>     at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:111)
>>>     at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:87)
>>>     at
>>>
>>> com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.
>>> java:57)
>>>
>>>     at com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:991)
>>>     at
>>>
>>> com.ctc.wstx.sr.BasicStreamReader.readTextSecondary(BasicStreamReade
>>> r
>>> .java:4647)
>>>
>>>     at
>>>
>>> com.ctc.wstx.sr.BasicStreamReader.finishToken(BasicStreamReader.java:
>>> 3721)
>>>
>>>     at
>>>
>>> com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.
>>> j
>>> ava:3675)
>>>
>>>     at
>>>
>>> com.ctc.wstx.sr.BasicStreamReader.getTextLength(BasicStreamReader.ja
>>> v
>>> a:907)
>>>
>>>     at
>>>
>>> org.apache.chemistry.opencmis.server.impl.atompub.AtomEntryParser.re
>>> a
>>> dBase64(AtomEntryParser.java:461)
>>>
>>>     at
>>>
>>> org.apache.chemistry.opencmis.server.impl.atompub.AtomEntryParser.pa
>>> r
>>> seCmisContent(AtomEntryParser.java:375)
>>>
>>>     at
>>>
>>> org.apache.chemistry.opencmis.server.impl.atompub.AtomEntryParser.pa
>>> r
>>> seEntry(AtomEntryParser.java:251)
>>>
>>>     at
>>>
>>> org.apache.chemistry.opencmis.server.impl.atompub.AtomEntryParser.pa
>>> r
>>> se(AtomEntryParser.java:214)
>>>
>>>     at
>>>
>>> org.apache.chemistry.opencmis.server.impl.atompub.ObjectService$Crea
>>> t
>>> e.serve(ObjectService.java:97)
>>>
>>>     at
>>>
>>> org.apache.chemistry.opencmis.server.shared.Dispatcher.dispatch(Disp
>>> a
>>> tcher.java:88)
>>>
>>>     at
>>>
>>> org.apache.chemistry.opencmis.server.impl.atompub.CmisAtomPubServlet.
>>> dispatch(CmisAtomPubServlet.java:292)
>>>
>>>     at
>>>
>>> org.apache.chemistry.opencmis.server.impl.atompub.CmisAtomPubServlet.
>>> service(CmisAtomPubServlet.java:228)
>>>
>>>     at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>>>     at
>>>
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(App
>>> l
>>> icationFilterChain.java:290)
>>>
>>>     at
>>>
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(Application
>>> F
>>> ilterChain.java:206)
>>>
>>>     at
>>> com.xerox.docushare.amber.util.UTF8Filter.doFilter(UTF8Filter.java:28)
>>>     at
>>>
>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(App
>>> l
>>> icationFilterChain.java:235)
>>>
>>>     at
>>>
>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(Application
>>> F
>>> ilterChain.java:206)
>>>
>>>     at
>>>
>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapper
>>> V
>>> alve.java:233)
>>>
>>>     at
>>>
>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContext
>>> V
>>> alve.java:191)
>>>
>>>     at
>>>
>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.
>>> j
>>> ava:127)
>>>
>>>     at
>>>
>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.
>>> j
>>> ava:102)
>>>
>>>     at
>>>
>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineVa
>>> l
>>> ve.java:109)
>>>
>>>     at
>>>
>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.ja
>>> v
>>> a:293)
>>>
>>>     at
>>>
>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.jav
>>> a
>>> :859)
>>>
>>>     at
>>>
>>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.proc
>>> e
>>> ss(Http11Protocol.java:602)
>>>
>>>     at
>>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
>>>     at java.lang.Thread.run(Thread.java:662)
>>>
>>> -----Original Message-----
>>> From: Florian Müller [mailto:fmui@apache.org]
>>> Sent: Thursday, March 27, 2014 4:46 PM
>>> To: Inouye, Brian
>>> Cc: dev@chemistry.apache.org
>>> Subject: Re: create service with a large content stream causes a 
>>> "Limit exceeded!" exception
>>>
>>> One additional remark:
>>> To avoid the Base64 encoding create an empty document and then set 
>>> the content. It's much more efficient when you have to handle 
>>> documents of that size.
>>> Or use the Browser binding.
>>>
>>>
>>> - Florian
>>>
>>>
>>>> Hi Florian,
>>>>
>>>> The Client is sending the content stream embedded in the 
>>>> cmisra:content element, so the resulting XML is huge.
>>>>
>>>> <?xml version="1.0" encoding="UTF-8" ?> <entry 
>>>> xmlns="http://www.w3.org/2005/Atom"
>>>> xmlns:app="http://www.w3.org/2007/app"
>>>> xmlns:cmisra="http://docs.oasis-open.org/ns/cmis/restatom/200908/">
>>>> <cmisra:content>
>>>> <cmisra:mediatype>video/quicktime</cmisra:mediatype>
>>>> <cmisra:base64>[base64 encoding of
>>>> file]</cmisra:base64></cmisra:content>
>>>> <cmisra:object
>>>> xmlns:cmis="http://docs.oasis-open.org/ns/cmis/core/200908/">
>>>> <cmis:properties>...</cmis:properties>
>>>> </cmisra:object>
>>>> <title>Demo.mov</title>
>>>> </entry>
>>>>
>>>> ... Brian ...
>>>> Brian Inouye, Xerox Corporation
>>>>
>>>> -----Original Message-----
>>>> From: Florian Müller [mailto:fmui@apache.org]
>>>> Sent: Thursday, March 27, 2014 3:03 PM
>>>> To: Inouye, Brian
>>>> Cc: dev@chemistry.apache.org
>>>> Subject: Re: create service with a large content stream causes a 
>>>> "Limit exceeded!" exception
>>>>
>>>> Hi Brian,
>>>>
>>>> The CappedInputStream does not limit the content stream, but the 
>>>> "envelope" around it. In case of the AtomPub binding it is limiting 
>>>> the size of the XML, but doesn't count the embedded document content.
>>>> Could you check how the request is created and if it contains a big 
>>>> XML portion?
>>>>
>>>>
>>>> - Florian
>>>>
>>>>
>>>>> Hi,
>>>>>
>>>>> I've run into a limitation in my CMIS Provider which uses OpenCMIS 
>>>>> 0.10.0.  When a Client sends a create request to my CMIS Provider 
>>>>> and the accompanying content stream is large, say 1.4 GB, 
>>>>> CappedInputStream raises an exception 
>>>>> CmisInvalidArgumentException("Limit exceeded!").
>>>>> AtomEntryParser.java creates the CappedInputStream object, passing 
>>>>> in a constant MAX_STREAM_LENGTH which is set to 10 * 1024 * 1024.
>>>>>
>>>>> public class CappedInputStream extends InputStream {
>>>>>     ...
>>>>>     private void checkLength() throws IOException {
>>>>>         if (counter > max) {
>>>>>             throw new CmisInvalidArgumentException("Limit exceeded!");
>>>>>         }
>>>>>     }
>>>>>     ...
>>>>> }
>>>>>
>>>>> public class AtomEntryParser {
>>>>>     ...
>>>>>     private static final long MAX_STREAM_LENGTH = 10 * 1024 * 1024;
>>>>>     ...
>>>>>     public void parse(InputStream stream) throws 
>>>>> XMLStreamException, IOException {
>>>>>         ...
>>>>>         cappedStream = new CappedInputStream(stream, 
>>>>> MAX_STREAM_LENGTH);
>>>>>         ...
>>>>>     }
>>>>> }
>>>>>
>>>>> What can I do to prevent this exception from occurring?  I tried 
>>>>> doubling the value, but the exception still occurs.
>>>>>
>>>>> ... Brian ...
>>>>> Brian Inouye, Xerox Corporation
>>>>>
>>>>>
>>


Mime
View raw message