camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vova Shelgunov <vvs...@gmail.com>
Subject Re: java.io.IOException: Response header too large during upload to camel-jetty endpoint
Date Wed, 23 Nov 2016 09:47:17 GMT
I have a case when I need to upload archives, size might be 10GB.
I think incoming file should not be saved in memory, but on disk (as tomcat
does).

>> Also mind there is something I think about not all web browsers / http
clients supporting big HTTP headers.

I do not send files from browser, they are sent using google-http-client
from another component of application.

2016-11-23 12:10 GMT+03:00 Claus Ibsen <claus.ibsen@gmail.com>:

> You can configure the header sizes also, you need to set them to higher
> values
> https://github.com/apache/camel/blob/master/components/
> camel-jetty9/src/main/docs/jetty-component.adoc
>
> Also mind there is something I think about not all web browsers / http
> clients supporting big HTTP headers.
>
> On Sun, Nov 20, 2016 at 4:36 PM, Vova Shelgunov <vvshvv@gmail.com> wrote:
>
> > Hi,
> >
> > I have application with the following jetty configuration:
> >
> > <route id="ds-rest-archive-upload-processor">
> > <from uri="jetty:https://storage:8443/data/archive?sslContextParam
> > etersRef=sslContextParameters&amp;requestBufferSize=32768&
> > amp;responseBufferSize=32768&amp;httpMethodRestrict=POST" />
> >
> > <to uri="bean:uploadFromArchiveFileProcessor?method=process"
> > pattern="InOut" />
> >
> > </route>
> >
> > <bean id="jetty" class="org.apache.camel.compon
> > ent.jetty9.JettyHttpComponent9">
> > <property name="endpointClass" value="org.apache.camel.component.jetty9.
> JettyHttpEndpoint9"
> > />
> > <property name="sslContextParameters" ref="sslContextParameters" />
> > <property name="useContinuation" value="true" />
> > <property name="requestBufferSize" value="32768" />
> > <property name="responseBufferSize" value="32768" />
> >
> > <property name="threadPool">
> > <bean class="org.eclipse.jetty.util.thread.QueuedThreadPool">
> > <constructor-arg index="0" value="1000" />
> > </bean>
> > </property>
> >
> > <property name="jettyHttpBinding">
> > <bean class="org.apache.camel.component.jetty.DefaultJettyHttpBinding">
> > <property name="transferException" value="false" />
> > </bean>
> > </property>
> > </bean>
> >
> >
> > *I am sending a file to this endpoint.When exception happens inside
> > UploadFromArchiveFileProcessor.process I see the following warning **in
> > the log**:*
> >
> > 2016-11-20 14:15:30,630 WARN  in org.eclipse.jetty.server.HttpC
> > hannel$CommitCallback.failed(HttpChannel.java:858) - Commit failed
> > java.io.IOException: Response header too large
> > at org.eclipse.jetty.http.HttpGenerator.generateResponse(HttpGe
> > nerator.java:402)
> > at org.eclipse.jetty.server.HttpConnection$SendCallback.process
> > (HttpConnection.java:655)
> > at org.eclipse.jetty.util.IteratingCallback.processing(Iteratin
> > gCallback.java:246)
> > at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCa
> > llback.java:208)
> > at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:471)
> > at org.eclipse.jetty.server.HttpChannel.sendResponse(
> HttpChannel.java:763)
> > at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:801)
> > at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:147)
> > at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:140)
> > at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:383)
> > at java.io.ByteArrayOutputStream.writeTo(ByteArrayOutputStream.java:167)
> > at org.eclipse.jetty.server.Utf8HttpWriter.write(
> Utf8HttpWriter.java:183)
> > at org.eclipse.jetty.server.HttpWriter.write(HttpWriter.java:71)
> > at java.io.PrintWriter.write(PrintWriter.java:456)
> > at java.io.PrintWriter.write(PrintWriter.java:473)
> > at java.io.PrintWriter.print(PrintWriter.java:603)
> > at org.apache.camel.http.common.DefaultHttpBinding.doWriteDirec
> > tResponse(DefaultHttpBinding.java:506)
> > at org.apache.camel.http.common.DefaultHttpBinding.doWriteRespo
> > nse(DefaultHttpBinding.java:390)
> > at org.apache.camel.http.common.DefaultHttpBinding.writeRespons
> > e(DefaultHttpBinding.java:322)
> > at org.apache.camel.component.jetty.CamelContinuationServlet.do
> > Service(CamelContinuationServlet.java:227)
> > at org.apache.camel.http.common.CamelServlet.service(
> CamelServlet.java:74)
> > at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> > at org.eclipse.jetty.servlet.ServletHolder.handle(
> ServletHolder.java:812)
> > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilte
> > r(ServletHandler.java:1669)
> > at org.apache.camel.component.jetty.CamelFilterWrapper.doFilter
> > (CamelFilterWrapper.java:45)
> > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilte
> > r(ServletHandler.java:1652)
> > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHan
> > dler.java:585)
> > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(
> > ContextHandler.java:1127)
> > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHand
> > ler.java:515)
> > at org.eclipse.jetty.server.handler.ContextHandler.doScope(
> > ContextHandler.java:1061)
> > at org.eclipse.jetty.server.handler.ScopedHandler.handle(Scoped
> > Handler.java:141)
> > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(Handl
> > erWrapper.java:97)
> > at org.eclipse.jetty.server.Server.handleAsync(Server.java:549)
> > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:318)
> > at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConne
> > ction.java:257)
> > at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnec
> > tion.java:544)
> > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(Queued
> > ThreadPool.java:635)
> > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedT
> > hreadPool.java:555)
> > at java.lang.Thread.run(Thread.java:745)
> > Caused by: java.nio.BufferOverflowException
> > at java.nio.Buffer.nextPutIndex(Buffer.java:521)
> > at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:169)
> > at org.eclipse.jetty.http.HttpGenerator.putSanitisedValue(HttpG
> > enerator.java:1066)
> > at org.eclipse.jetty.http.HttpGenerator.putTo(HttpGenerator.java:1088)
> > at org.eclipse.jetty.http.HttpGenerator.generateHeaders(HttpGen
> > erator.java:705)
> > at org.eclipse.jetty.http.HttpGenerator.generateResponse(HttpGe
> > nerator.java:387)
> > ... 38 more
> >
> > *When an exception does not occur and I do not add the following code to
> > finally block of process method, I will see the same exception and route
> > will fail.*
> >
> > message.removeHeader(attachment.getName());
> >
> > *I tried to debug HttpGenerator.generateHeaders and see that inside
> > _info._httpFields there are following fields:*
> >
> >
> > ‚Äč
> > I do not know what file from the request is doing here, but it is strange
> > at least.
> >
> > Could you please say how to avoid such exceptions?
> > Also strange thing that file content is stored in memory, so I am not
> able
> > to upload large file.
> >
> >
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message