chemistry-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Florian Müller <f...@apache.org>
Subject RE: create service with a large content stream causes a "Limit exceeded!" exception
Date Wed, 02 Apr 2014 08:42:25 GMT
 Hi Brian,

 A trace would be great.
 It's good to know that the Workbench upload works. That focuses the 
 investigation on the incoming XML.


 - Florian



> Hi Florian,
>
> This will be rather difficult to reproduce.  The "Limit exceeded!"
> server exception occurs when the file is being uploaded from a
> particular Mobile App going through WiFi to my server.  However, the
> exception does not occur if the same file is uploaded from the CMIS
> Workbench to my server.  I hope this isn't a timing-sensitive 
> problem.
> I am going to look at a packet trace to see if I can see any
> differences between how the Mobile App is transferring the file vs.
> the CMIS Workbench.
>
> ... Brian ...
>
> -----Original Message-----
> From: Florian Müller [mailto:fmui@apache.org]
> Sent: Monday, March 31, 2014 12:31 PM
> To: Inouye, Brian
> Cc: dev@chemistry.apache.org
> Subject: Re: create service with a large content stream causes a
> "Limit exceeded!" exception
>
> Hi Brian,
>
> I tried a lot but I cannot reproduce it.
> Is it possible to reproduce it with a smaller content and send me the
> request? I would like to debug this.
>
>
> - Florian
>
>
>> Hi Florian,
>>
>> No, we did not change anything in the OpenCMIS code.
>>
>> AtomEntryParser uses XMLUtils to create the parser:
>>
>> 	XMLStreamReader parser = XMLUtils.createParser(cappedStream);
>>
>> XMLUtils explicitly sets isCoalescing to FALSE.
>>
>>         XML_INPUT_FACTORY.setProperty(XMLInputFactory.IS_COALESCING,
>> Boolean.FALSE);
>>
>> ... Brian ...
>>
>> -----Original Message-----
>> From: Florian Müller [mailto:fmui@apache.org]
>> Sent: Friday, March 28, 2014 1:38 PM
>> To: Inouye, Brian
>> Cc: dev@chemistry.apache.org
>> Subject: Re: create service with a large content stream causes a
>> "Limit exceeded!" exception
>>
>> Hi Brian,
>>
>> I've revalidated that uploading huge documents works as expected.
>>
>> What you are seeing is only possible if the property 
>> "javax.xml.stream.isCoalescing" at the XMLInputFactory has been set to 
>> TRUE somehow. Did you change anything in the OpenCMIS code?
>>
>>
>> - Florian
>>
>>
>>> Hi Brian,
>>>
>>> The parser should process the the content of base64 tag in small 
>>> chunks.
>>> For some reasons it tries to read much more than that in your 
>>> setup.
>>> I'll look into it.
>>>
>>> - Florian
>>>
>>>
>>>> Hi Florian,
>>>>
>>>> Here is the exception stack.  It looks like it's using the 
>>>> Woodstox
>>>> parser (I assume that's what com.ctc.wstx is).  The exception is
>>>> happening when AtomEntryParser is calling readBase64, so I think 
>>>> it
>>>> doesn't like the amount of the base64 encoded content stream.  The
>>>> rest of the XML, excluding the content stream, is very small.
>>>>
>>>> We don't have control over the Client so our options are limited.
>>>>
>>>> ... Brian ...
>>>>
>>>>
>>>> 
>>>> org.apache.chemistry.opencmis.commons.exceptions.CmisInvalidArgumentException:
>>>>
>>>> Limit exceeded!
>>>>     at
>>>>
>>>> 
>>>> org.apache.chemistry.opencmis.server.shared.CappedInputStream.checkL
>>>> e
>>>> ngth(CappedInputStream.java:71)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.chemistry.opencmis.server.shared.CappedInputStream.read(C
>>>> a
>>>> ppedInputStream.java:107)
>>>>
>>>>     at com.ctc.wstx.io.BaseReader.readBytes(BaseReader.java:155)
>>>>     at com.ctc.wstx.io.UTF8Reader.loadMore(UTF8Reader.java:368)
>>>>     at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:111)
>>>>     at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:87)
>>>>     at
>>>>
>>>> 
>>>> com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.
>>>> java:57)
>>>>
>>>>     at 
>>>> com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:991)
>>>>     at
>>>>
>>>> 
>>>> com.ctc.wstx.sr.BasicStreamReader.readTextSecondary(BasicStreamReade
>>>> r
>>>> .java:4647)
>>>>
>>>>     at
>>>>
>>>> 
>>>> com.ctc.wstx.sr.BasicStreamReader.finishToken(BasicStreamReader.java:
>>>> 3721)
>>>>
>>>>     at
>>>>
>>>> 
>>>> com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.
>>>> j
>>>> ava:3675)
>>>>
>>>>     at
>>>>
>>>> 
>>>> com.ctc.wstx.sr.BasicStreamReader.getTextLength(BasicStreamReader.ja
>>>> v
>>>> a:907)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.chemistry.opencmis.server.impl.atompub.AtomEntryParser.re
>>>> a
>>>> dBase64(AtomEntryParser.java:461)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.chemistry.opencmis.server.impl.atompub.AtomEntryParser.pa
>>>> r
>>>> seCmisContent(AtomEntryParser.java:375)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.chemistry.opencmis.server.impl.atompub.AtomEntryParser.pa
>>>> r
>>>> seEntry(AtomEntryParser.java:251)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.chemistry.opencmis.server.impl.atompub.AtomEntryParser.pa
>>>> r
>>>> se(AtomEntryParser.java:214)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.chemistry.opencmis.server.impl.atompub.ObjectService$Crea
>>>> t
>>>> e.serve(ObjectService.java:97)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.chemistry.opencmis.server.shared.Dispatcher.dispatch(Disp
>>>> a
>>>> tcher.java:88)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.chemistry.opencmis.server.impl.atompub.CmisAtomPubServlet.
>>>> dispatch(CmisAtomPubServlet.java:292)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.chemistry.opencmis.server.impl.atompub.CmisAtomPubServlet.
>>>> service(CmisAtomPubServlet.java:228)
>>>>
>>>>     at 
>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>>>>     at
>>>>
>>>> 
>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(App
>>>> l
>>>> icationFilterChain.java:290)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(Application
>>>> F
>>>> ilterChain.java:206)
>>>>
>>>>     at
>>>> 
>>>> com.xerox.docushare.amber.util.UTF8Filter.doFilter(UTF8Filter.java:28)
>>>>     at
>>>>
>>>> 
>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(App
>>>> l
>>>> icationFilterChain.java:235)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(Application
>>>> F
>>>> ilterChain.java:206)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapper
>>>> V
>>>> alve.java:233)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContext
>>>> V
>>>> alve.java:191)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.
>>>> j
>>>> ava:127)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.
>>>> j
>>>> ava:102)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineVa
>>>> l
>>>> ve.java:109)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.ja
>>>> v
>>>> a:293)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.jav
>>>> a
>>>> :859)
>>>>
>>>>     at
>>>>
>>>> 
>>>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.proc
>>>> e
>>>> ss(Http11Protocol.java:602)
>>>>
>>>>     at
>>>> 
>>>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
>>>>     at java.lang.Thread.run(Thread.java:662)
>>>>
>>>> -----Original Message-----
>>>> From: Florian Müller [mailto:fmui@apache.org]
>>>> Sent: Thursday, March 27, 2014 4:46 PM
>>>> To: Inouye, Brian
>>>> Cc: dev@chemistry.apache.org
>>>> Subject: Re: create service with a large content stream causes a
>>>> "Limit exceeded!" exception
>>>>
>>>> One additional remark:
>>>> To avoid the Base64 encoding create an empty document and then set
>>>> the content. It's much more efficient when you have to handle
>>>> documents of that size.
>>>> Or use the Browser binding.
>>>>
>>>>
>>>> - Florian
>>>>
>>>>
>>>>> Hi Florian,
>>>>>
>>>>> The Client is sending the content stream embedded in the
>>>>> cmisra:content element, so the resulting XML is huge.
>>>>>
>>>>> <?xml version="1.0" encoding="UTF-8" ?> <entry
>>>>> xmlns="http://www.w3.org/2005/Atom"
>>>>> xmlns:app="http://www.w3.org/2007/app"
>>>>> 
>>>>> xmlns:cmisra="http://docs.oasis-open.org/ns/cmis/restatom/200908/">
>>>>> <cmisra:content>
>>>>> <cmisra:mediatype>video/quicktime</cmisra:mediatype>
>>>>> <cmisra:base64>[base64 encoding of
>>>>> file]</cmisra:base64></cmisra:content>
>>>>> <cmisra:object
>>>>> xmlns:cmis="http://docs.oasis-open.org/ns/cmis/core/200908/">
>>>>> <cmis:properties>...</cmis:properties>
>>>>> </cmisra:object>
>>>>> <title>Demo.mov</title>
>>>>> </entry>
>>>>>
>>>>> ... Brian ...
>>>>> Brian Inouye, Xerox Corporation
>>>>>
>>>>> -----Original Message-----
>>>>> From: Florian Müller [mailto:fmui@apache.org]
>>>>> Sent: Thursday, March 27, 2014 3:03 PM
>>>>> To: Inouye, Brian
>>>>> Cc: dev@chemistry.apache.org
>>>>> Subject: Re: create service with a large content stream causes a
>>>>> "Limit exceeded!" exception
>>>>>
>>>>> Hi Brian,
>>>>>
>>>>> The CappedInputStream does not limit the content stream, but the
>>>>> "envelope" around it. In case of the AtomPub binding it is 
>>>>> limiting
>>>>> the size of the XML, but doesn't count the embedded document 
>>>>> content.
>>>>> Could you check how the request is created and if it contains a 
>>>>> big
>>>>> XML portion?
>>>>>
>>>>>
>>>>> - Florian
>>>>>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I've run into a limitation in my CMIS Provider which uses 
>>>>>> OpenCMIS
>>>>>> 0.10.0.  When a Client sends a create request to my CMIS 
>>>>>> Provider
>>>>>> and the accompanying content stream is large, say 1.4 GB,
>>>>>> CappedInputStream raises an exception
>>>>>> CmisInvalidArgumentException("Limit exceeded!").
>>>>>> AtomEntryParser.java creates the CappedInputStream object, 
>>>>>> passing
>>>>>> in a constant MAX_STREAM_LENGTH which is set to 10 * 1024 * 
>>>>>> 1024.
>>>>>>
>>>>>> public class CappedInputStream extends InputStream {
>>>>>>     ...
>>>>>>     private void checkLength() throws IOException {
>>>>>>         if (counter > max) {
>>>>>>             throw new CmisInvalidArgumentException("Limit 
>>>>>> exceeded!");
>>>>>>         }
>>>>>>     }
>>>>>>     ...
>>>>>> }
>>>>>>
>>>>>> public class AtomEntryParser {
>>>>>>     ...
>>>>>>     private static final long MAX_STREAM_LENGTH = 10 * 1024 * 
>>>>>> 1024;
>>>>>>     ...
>>>>>>     public void parse(InputStream stream) throws
>>>>>> XMLStreamException, IOException {
>>>>>>         ...
>>>>>>         cappedStream = new CappedInputStream(stream,
>>>>>> MAX_STREAM_LENGTH);
>>>>>>         ...
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>> What can I do to prevent this exception from occurring?  I tried
>>>>>> doubling the value, but the exception still occurs.
>>>>>>
>>>>>> ... Brian ...
>>>>>> Brian Inouye, Xerox Corporation
>>>>>>
>>>>>>
>>>


Mime
View raw message