hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Gordineer <joshg...@gmail.com>
Subject Handling (indefinitely) streaming sources
Date Sat, 24 Mar 2012 00:05:45 GMT
I have a question about how to protect against requests to streamed
sources.  A project background is that we allow execution on our servers of
outbound http requests based on user input.  So in essence we need to
protect against abuse by adding restrictions on response size/time etc.  We
have done this successfully in the past by adding a counter to the
inputstream to make sure data is below an arbitrary max size however, we
have come across a case when a user inputs (either incorrectly or
maliciously) tries to fetch an audio stream which causes our read thread to
continue endlessly reading the content (details of the actual feed and
stacktrace below).  This request hangs getting the HttpResponse from the
httpclient.execute(request) call.  Ideally we could read the response
header however since the client is hanging at the execute line I don't have
the handle to fetch the headers.

Basically we need identify these requests and kill them prior to causing
our machine to churn reading data.  My first idea is adding a hook in
httpclient to track the size of the request we are reading and kill it
after it gets past a threshold however it wasn't clear to me how to do so
with httpclient (I looked at creating a specialization of
DefaultResponseParser however I didn't see how to instantiate my version).

Any suggestions are welcome.  Thanks in advance!

--Josh

Detailed info:

$ curl -v http://scfire-dtc-aa04.stream.aol.com:80/stream/1030/7.html
* About to connect() to scfire-dtc-aa04.stream.aol.com port 80 (#0)
*   Trying 205.188.234.4... connected
* Connected to scfire-dtc-aa04.stream.aol.com (205.188.234.4) port 80 (#0)
> GET /stream/1030/7.html HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7
OpenSSL/0.9.8r zlib/1.2.3
> Host: scfire-dtc-aa04.stream.aol.com
> Accept: */*
>
ICY 200 OK
icy-notice1: <BR>This stream requires <a href="http://www.winamp.com/
">Winamp</a><BR>
icy-notice2: Firehose Ultravox/SHOUTcast Relay Server/Linux v2.6.0<BR>
icy-name: RADIOUP.COM - THE HITLIST (formely 108.fm) - #1 FOR ALL HIT MUSIC
icy-genre: Top 40 Pop Rap Hip Hop Top40
icy-url: http://www.radioup.com/
content-type: audio/mpeg
icy-pub: 1
icy-br: 128

Sstack dump for the read thread (versions httpclient-4.1.3.jar
 httpcore-4.1.4.jar):

"Instance-thread-1" prio=10 tid=0x89ae5c00 nid=0xe04 runnable [0x03785000]
   java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
at
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
at
org.apache.http.impl.conn.LoggingSessionInputBuffer.readLine(LoggingSessionInputBuffer.java:115)
at
org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
at
org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
<snip>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message