hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Olofsson <r...@khelekore.org>
Subject Re: Problem on creating web proxy based on ElementalHTTP Server
Date Fri, 23 Mar 2007 21:06:30 GMT

> I want to ask one more question, is my approach a right way to implement a
> header/content filtering-altering web proxy? 

Ugh!, I would like to ask why you would want to do this.

Speaking from experience of developing a filtering web proxy I can
say that doing it correctly is a huge task.
Of course depending on what features you include it may or may not
become easier.

Your proxy is very small and it may seem to work for a few requests
with a web browser, but it is far from correct. From what I can
see it does not handle the headers correctly like hop headers, version
headers, date headers. Since I do not follow the http components project
very close I can not say what the result of this misuse of http
headers will result in.

Here are a small unordered list of things you may want to include in
a filtering http proxy:
1) Ip based access level restrictions.
2) Proxy authentication,
3) Secure access, if you want people to use your proxy over the web.
4) Request header filtering.
5) Response header filtering.
6) Caching, skip this if you can, http rules for caching are very
7) Html parsing library, but note that you need some library that deals
    with real word html content. Trying to use an xml parser is not a
    good idea.
8) Compressing content, you have gzip in the standard java api, but
    depending on your needs and want to do nio-based transactions the
    standard libraries need some serious wrappers.
9) Other content filtering options like image recompression.
10) Persistent connections, both downstream and upstreams. Doing this
     towards the user is trivial, doing this  upstreams is kind of
     complex, you need to handle retransmission of requests.
11) proxy chaining support.
12) Restrictions on https proxying.
13) Streaming content, from both directions, not doing this leads to
     OOME and slow responses.
14) Thread pool handling, you do not want one 2 threads per connection
     (one for reading downstream channel, one for upstream).

What I can see in your code you have not implemented even one of the

When you filter content you must handle the request and response headers
with care, or you will seriously mess up some systems.

If you really care you can try to test your proxy with some automated
tool like the co-advisor test: http://coad.measurement-factory.com/
Using their service costs money, but you can do the free demo and
if you plan on making your proxy open source you can also ask them
to give you a free account.

If you just want to study http and have some fun, then building a
proxy can be ok. If you want to do it for real, then you need
to plan a few step ahead.

Also note that there are several java, but not http components,
based proxies out there like:
rabbit (my proxy that do all of the above), muffin (I think it is a
dead project), paw, jigsaw, ...

Most of the above are open source in some way

Have fun


To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org

View raw message