Mailing-List: contact users-help@camel.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@camel.apache.org
MIME-Version: 1.0
In-Reply-To: <308255934.778948.1476880108991@mail.yahoo.com>
References: <CABEDHJZigNoXsY3ooCj8KfmdkTb2ExqEkAAt+4dZU96dtGV7ag@mail.gmail.com>
 <308255934.778948.1476880108991@mail.yahoo.com>
From: Vova Shelgunov <vvshvv@gmail.com>
Date: Wed, 19 Oct 2016 15:39:39 +0300
Message-ID: <CABEDHJadK_jnx_q98m+_8jTiUruseDDGUMm1eQNWJC3iywuo4Q@mail.gmail.com>
Subject: Re: OutOfMemoryError: Java heap space when sending large file to
 jetty endpoint
To: users@camel.apache.org, Andrea Cosentino <ancosen1985@yahoo.com>
Content-Type: multipart/alternative; boundary=f403045de53c414813053f371af9
archived-at: Wed, 19 Oct 2016 12:39:52 -0000

--f403045de53c414813053f371af9
Content-Type: text/plain; charset=UTF-8

Hi Andrea,

Yes, I run application using jar file. Java has 1G heap.

Actually I assumed that camel will not read file in single string, but will
use InputStream to read the data.

2016-10-19 15:28 GMT+03:00 Andrea Cosentino <ancosen1985@yahoo.com.invalid>:

> Is the camel-context deployed as a simple JAR? What are you using inside
> the Debian container? How much memory has your container?
> Did you set JVM_OPTS witch specific values for max Heap size and min Heap
> size?
>
>  --
> Andrea Cosentino
> ----------------------------------
> Apache Camel PMC Member
> Apache Karaf Committer
> Apache Servicemix Committer
> Email: ancosen1985@yahoo.com
> Twitter: @oscerd2
> Github: oscerd
>
>
>
> On Wednesday, October 19, 2016 2:00 PM, Vova Shelgunov <vvshvv@gmail.com>
> wrote:
> Hi,
>
> I catched this OutOfMemoryError when I sent a 250 MB file to jetty
> endpoint:
>
> java.lang.OutOfMemoryError: Java heap space
>       at java.lang.StringCoding.decode(StringCoding.java:215)
>       at java.lang.String.<init>(String.java:463)
>       at java.lang.String.<init>(String.java:515)
>       at
> org.eclipse.jetty.servlets.MultiPartFilter$Wrapper.
> getParameterBytesAsString(MultiPartFilter.java:390)
>       at
> org.eclipse.jetty.servlets.MultiPartFilter$Wrapper.getParameterValues(
> MultiPartFilter.java:337)
>       at
> org.apache.camel.http.common.DefaultHttpBinding.populateRequestParameters(
> DefaultHttpBinding.java:219)
>       at
> org.apache.camel.http.common.DefaultHttpBinding.readHeaders(
> DefaultHttpBinding.java:174)
>       at
> org.apache.camel.http.common.DefaultHttpBinding.readRequest(
> DefaultHttpBinding.java:110)
>       at
> org.apache.camel.http.common.HttpMessage.<init>(HttpMessage.java:52)
>       at
> org.apache.camel.component.jetty.CamelContinuationServlet.doService(
> CamelContinuationServlet.java:161)
>       at
> org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:74)
>       at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
>       at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
>       at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> doFilter(ServletHandler.java:1669)
>       at
> org.eclipse.jetty.servlets.MultiPartFilter.doFilter(
> MultiPartFilter.java:200)
>       at
> org.apache.camel.component.jetty.CamelFilterWrapper.
> doFilter(CamelFilterWrapper.java:43)
>       at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> doFilter(ServletHandler.java:1652)
>       at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>       at
> org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1127)
>       at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>       at
> org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:1061)
>       at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:141)
>       at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:97)
>       at org.eclipse.jetty.server.Server.handle(Server.java:499)
>       at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
>       at
> org.eclipse.jetty.server.HttpConnection.onFillable(
> HttpConnection.java:257)
>       at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
>       at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> QueuedThreadPool.java:635)
>       at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(
> QueuedThreadPool.java:555)
>       at java.lang.Thread.run(Thread.java:745)
>
> My configuration:
>
> <route id="ds-rest-archive-upload-processor">
> <from uri="jetty:
> https://storage:8443/data/archive?sslContextParametersRef=
> sslContextParameters&requestBufferSize=32768&responseBufferSize=32768&
> httpMethodRestrict=POST
> <https://storage:8443/data/archive?sslContextParametersRef=
> sslContextParameters&requestBufferSize=32768&responseBufferSize=32768&
> httpMethodRestrict=POST>"
> />
>
> <to uri="bean:uploadFromArchiveFileProcessor?method=process"
> pattern="InOut" />
>
> </route>
>
> <bean id="jetty"
> class="org.apache.camel.component.jetty9.JettyHttpComponent9">
> <property name="endpointClass"
> value="org.apache.camel.component.jetty9.JettyHttpEndpoint9" />
> <property name="sslContextParameters" ref="sslContextParameters" />
> <property name="useContinuation" value="true" />
> <property name="requestBufferSize" value="32768" />
> <property name="responseBufferSize" value="32768" />
>
> <property name="threadPool">
> <bean class="org.eclipse.jetty.util.thread.QueuedThreadPool">
> <constructor-arg index="0" value="1000" />
> </bean>
> </property>
>
> <property name="jettyHttpBinding">
> <bean class="org.apache.camel.component.jetty.DefaultJettyHttpBinding">
> <property name="transferException" value="false" />
> </bean>
> </property>
> </bean>
>
> My camel application is running in docker container with limited amount of
> memory, and I did not expect, that a file will be read a string instead of
> reading from InputStream.
>
> Is there any way to consume large files with small amount of memory?
>
>
> Regards,
>
> Uladzimir
>

--f403045de53c414813053f371af9--