Mailing-List: contact users-help@camel.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@camel.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Date: Thu, 24 Jul 2014 23:35:46 -0700 (PDT)
From: rsteppac2 <ralf@steppacher.name>
To: users@camel.apache.org
Message-ID: <1406270146750-5754424.post@n5.nabble.com>
Subject: Stream-only reverse proxy with minimal memory footprint
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I am trying to implement a memory efficient http reverse proxy that is
working on streams only to handle large HTTP requests and responses. This
should be as easy as described here:

http://camel.apache.org/how-to-use-camel-as-a-http-proxy-between-a-client-and-server.html 

The Jetty consumer places the socket input stream with the http POST body
into the exchange and I can hook it up with a producer to forward the
request. No problem here.

The problem is the response. All http producers that I am aware of (Jetty,
http4, netty-http) read the response stream into a byte-array,wrap it in an
InputStream and place that in the exchange body. Thus a large http response
blows my proxy out of the water. None of the producers seem to offer an
option to make them put the original socket input stream into the exchange,
just as the Jetty consumer does.

http4: Everything happens in
org.apache.camel.component.http4.HttpProducer::process() ->
populateResponse(..) -> extractResponseBody(..) ->
doExtractResponseBodyAsStream(); here the original stream is copied into an
instance of CachedOutputStream, backed by a byte-array.

Jetty: org.eclipse.jetty.client.AsyncHttpConnection::handle() ->
org.eclipse.jetty.http.HttpParser::parseNext() will fill a byte-array in
org.eclipse.jetty.client.ContentExchange which is a CachedExchange which is
a HttpExchange.

netty-http: Builds a pipeline that assembles the HttpResponse content as a
composite ChannelBuffer. The wrapped channel buffers make up the complete
response stream.

Is there a way to configure any of the clients to place the socket input
stream into the exchange body?

This thread describes the same problem and proposes a patch, which I think
has not made it into the Camel code base as of yet:
http://camel.465427.n5.nabble.com/Chunking-issue-with-http-producer-td5735075.html

The question is how to close the socket stream of the client. If I should
have to replicate the suggested patch, would a Synchronization call-back
registered with the exchange via Exchange::addOnCompletion(Synchronization)
be the correct place to do it? I have tried with a FileInputStream and that
seems to work. I.e. the call-back is triggered after the stream contents
have been written out to the socket.


Thanks!
Ralf


--
View this message in context: http://camel.465427.n5.nabble.com/Stream-only-reverse-proxy-with-minimal-memory-footprint-tp5754424.html
Sent from the Camel - Users mailing list archive at Nabble.com.