camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomohisa Igarashi <tm.igara...@gmail.com>
Subject Re: OutOfMemoryError: Java heap space when sending large file to jetty endpoint
Date Thu, 20 Oct 2016 01:24:03 GMT
Hi,

It seems like the OOM occurs when it's mapping request parameters, but not request body which
is passed as an InputStream by default. Do you have any large parameter other than body?
Note that if you enable allowJavaSerializedObject, then body is deseriallized as a Java Object,
may cause OOM.

Thanks,
Tomo

On 10/19/2016 09:39 PM, Vova Shelgunov wrote:
> Hi Andrea,
>
> Yes, I run application using jar file. Java has 1G heap.
>
> Actually I assumed that camel will not read file in single string, but will
> use InputStream to read the data.
>
> 2016-10-19 15:28 GMT+03:00 Andrea Cosentino <ancosen1985@yahoo.com.invalid>:
>
>> Is the camel-context deployed as a simple JAR? What are you using inside
>> the Debian container? How much memory has your container?
>> Did you set JVM_OPTS witch specific values for max Heap size and min Heap
>> size?
>>
>>  --
>> Andrea Cosentino
>> ----------------------------------
>> Apache Camel PMC Member
>> Apache Karaf Committer
>> Apache Servicemix Committer
>> Email: ancosen1985@yahoo.com
>> Twitter: @oscerd2
>> Github: oscerd
>>
>>
>>
>> On Wednesday, October 19, 2016 2:00 PM, Vova Shelgunov <vvshvv@gmail.com>
>> wrote:
>> Hi,
>>
>> I catched this OutOfMemoryError when I sent a 250 MB file to jetty
>> endpoint:
>>
>> java.lang.OutOfMemoryError: Java heap space
>>       at java.lang.StringCoding.decode(StringCoding.java:215)
>>       at java.lang.String.<init>(String.java:463)
>>       at java.lang.String.<init>(String.java:515)
>>       at
>> org.eclipse.jetty.servlets.MultiPartFilter$Wrapper.
>> getParameterBytesAsString(MultiPartFilter.java:390)
>>       at
>> org.eclipse.jetty.servlets.MultiPartFilter$Wrapper.getParameterValues(
>> MultiPartFilter.java:337)
>>       at
>> org.apache.camel.http.common.DefaultHttpBinding.populateRequestParameters(
>> DefaultHttpBinding.java:219)
>>       at
>> org.apache.camel.http.common.DefaultHttpBinding.readHeaders(
>> DefaultHttpBinding.java:174)
>>       at
>> org.apache.camel.http.common.DefaultHttpBinding.readRequest(
>> DefaultHttpBinding.java:110)
>>       at
>> org.apache.camel.http.common.HttpMessage.<init>(HttpMessage.java:52)
>>       at
>> org.apache.camel.component.jetty.CamelContinuationServlet.doService(
>> CamelContinuationServlet.java:161)
>>       at
>> org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:74)
>>       at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>>       at
>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
>>       at
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.
>> doFilter(ServletHandler.java:1669)
>>       at
>> org.eclipse.jetty.servlets.MultiPartFilter.doFilter(
>> MultiPartFilter.java:200)
>>       at
>> org.apache.camel.component.jetty.CamelFilterWrapper.
>> doFilter(CamelFilterWrapper.java:43)
>>       at
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.
>> doFilter(ServletHandler.java:1652)
>>       at
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>>       at
>> org.eclipse.jetty.server.handler.ContextHandler.
>> doHandle(ContextHandler.java:1127)
>>       at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>>       at
>> org.eclipse.jetty.server.handler.ContextHandler.
>> doScope(ContextHandler.java:1061)
>>       at
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(
>> ScopedHandler.java:141)
>>       at
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>> HandlerWrapper.java:97)
>>       at org.eclipse.jetty.server.Server.handle(Server.java:499)
>>       at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
>>       at
>> org.eclipse.jetty.server.HttpConnection.onFillable(
>> HttpConnection.java:257)
>>       at
>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
>>       at
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
>> QueuedThreadPool.java:635)
>>       at
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(
>> QueuedThreadPool.java:555)
>>       at java.lang.Thread.run(Thread.java:745)
>>
>> My configuration:
>>
>> <route id="ds-rest-archive-upload-processor">
>> <from uri="jetty:
>> https://storage:8443/data/archive?sslContextParametersRef=
>> sslContextParameters&requestBufferSize=32768&responseBufferSize=32768&
>> httpMethodRestrict=POST
>> <https://storage:8443/data/archive?sslContextParametersRef=
>> sslContextParameters&requestBufferSize=32768&responseBufferSize=32768&
>> httpMethodRestrict=POST>"
>> />
>>
>> <to uri="bean:uploadFromArchiveFileProcessor?method=process"
>> pattern="InOut" />
>>
>> </route>
>>
>> <bean id="jetty"
>> class="org.apache.camel.component.jetty9.JettyHttpComponent9">
>> <property name="endpointClass"
>> value="org.apache.camel.component.jetty9.JettyHttpEndpoint9" />
>> <property name="sslContextParameters" ref="sslContextParameters" />
>> <property name="useContinuation" value="true" />
>> <property name="requestBufferSize" value="32768" />
>> <property name="responseBufferSize" value="32768" />
>>
>> <property name="threadPool">
>> <bean class="org.eclipse.jetty.util.thread.QueuedThreadPool">
>> <constructor-arg index="0" value="1000" />
>> </bean>
>> </property>
>>
>> <property name="jettyHttpBinding">
>> <bean class="org.apache.camel.component.jetty.DefaultJettyHttpBinding">
>> <property name="transferException" value="false" />
>> </bean>
>> </property>
>> </bean>
>>
>> My camel application is running in docker container with limited amount of
>> memory, and I did not expect, that a file will be read a string instead of
>> reading from InputStream.
>>
>> Is there any way to consume large files with small amount of memory?
>>
>>
>> Regards,
>>
>> Uladzimir
>>
>

Mime
View raw message