hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ken Krugler <kkrugler_li...@transpac.com>
Subject Re: Trying to send raw requests without URL encodings?
Date Sun, 03 Apr 2011 15:22:35 GMT

On Apr 3, 2011, at 8:14am, Sam Crawford wrote:

> Steve,
> 
> I've had a dig around the code this afternoon and understand the
> issues you've run into. The java.net.URI class is fairly well embedded
> throughout, and overriding HttpRequestBase.getRequestLine() alone will
> not be enough.
> 
> One thought: Have you considered copying and modifying the source for
> java.net.URI to not throw the URISyntaxException? (Sadly you can't
> just extend it, as it's declared final). Patching HttpClient /
> HttpCore to use your new class would be very straightforward.
> 
> You can take a look at some source for java.net.URI at
> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/net/URI.java/?v=source

Oddly enough, I was just looking at the same thing.

I'd run into a similar issue, though with hostnames that are allowed by DNS servers but rejected
by URI (e.g. "-sub.domain.com")

One issue I notice is that you might need to handle is having these query parameters re-encoded
when say URI.toURL() is called. E.g.

        java.net.URI olduri = new java.net.URI("http", "example.com", null, "var=<xss>",
null);
        System.out.println(olduri.toURL());

will print out "http://example.com?var=%3Cxss%3E"

-- Ken

> On 3 April 2011 06:00, Steve Labarbera <steve.labarbera@whitehatsec.com> wrote:
>> Thanks for the reply but actually that was me asking that question a few months back.
I got so frustrated I stepped away from the problem. Now I'm back determined to find a solution.
As what was mentioned in that response the HttpCore is built on URI and I have not found a
way to bypass the java.net.URISyntaxException: Illegal character in query at index exception
that is thrown when illegal characters are added to the query.
>> 
>> So, as an example, I need to send a request like http://example.com?var=<xss>
<====== not url encoded
>> 
>> Problem is no matter what I try the URI class throws an exception. I'm trying to
avoid having to have to re-invent the wheel and build my own HTTPClient class.You think I
have another option?
>> 
>> Thanks in advance for the help,
>> 
>> Steve
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> _____
>> From: Sam Crawford [samcrawford@gmail.com]
>> Sent: Saturday, April 02, 2011 3:00 PM
>> To: HttpClient User Discussion
>> Cc: Steve Labarbera
>> Subject: Re: Trying to send raw requests without URL encodings?
>> 
>> Hi Steve,
>> 
>> A very similar question was asked here:
>> http://old.nabble.com/Apache's-HttpClient-don't-want-URL-Encoding-any-ideas-how-I-can-bypass--td25762297.html
>> 
>> This only deals with the request URL, but it would mean you don't need
>> to move to HttpCore.
>> 
>> Implementing your own RequestEntity makes the most sense to me for
>> handling the POST scenario.
>> 
>> Hope this helps,
>> 
>> Sam
>> 
>> 
>> 
>> On 2 April 2011 22:15, Steve Labarbera <steve.labarbera@whitehatsec.com> wrote:
>>> I'm trying to send requests out without any URL encoding. This is necessary because
we need to simulate sending requests bypassing the browser encoding the parameters. I have
done a bunch of reading and it appears that one user mentioned implementing my own RequestEntity
class. Is this the best option?
>>> 
>>> - Ideally, I need to be able to send anything in the url or post parameters without
any URL encoding being done on them before being sent to the server.
>>> 
>>>  - Would HttpClient be best approach or will I need to use a lower level library
like HttpCore.
>>> 
>>> Any input that can help me move forward would be very helpful.
>>> 
>>> Thanks,
>>> 
>>> Steve
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>> 
>>> 
>> 
>> ________________________________________
>> From: Sam Crawford [samcrawford@gmail.com]
>> Sent: Saturday, April 02, 2011 3:00 PM
>> To: HttpClient User Discussion
>> Cc: Steve Labarbera
>> Subject: Re: Trying to send raw requests without URL encodings?
>> 
>> Hi Steve,
>> 
>> A very similar question was asked here:
>> http://old.nabble.com/Apache's-HttpClient-don't-want-URL-Encoding-any-ideas-how-I-can-bypass--td25762297.html
>> 
>> This only deals with the request URL, but it would mean you don't need
>> to move to HttpCore.
>> 
>> Implementing your own RequestEntity makes the most sense to me for
>> handling the POST scenario.
>> 
>> Hope this helps,
>> 
>> Sam
>> 
>> 
>> 
>> On 2 April 2011 22:15, Steve Labarbera <steve.labarbera@whitehatsec.com> wrote:
>>> I'm trying to send requests out without any URL encoding. This is necessary because
we need to simulate sending requests bypassing the browser encoding the parameters. I have
done a bunch of reading and it appears that one user mentioned implementing my own RequestEntity
class. Is this the best option?
>>> 
>>> - Ideally, I need to be able to send anything in the url or post parameters without
any URL encoding being done on them before being sent to the server.
>>> 
>>>  - Would HttpClient be best approach or will I need to use a lower level library
like HttpCore.
>>> 
>>> Any input that can help me move forward would be very helpful.
>>> 
>>> Thanks,
>>> 
>>> Steve
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>>> 
>>> 
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> 

--------------------------
Ken Krugler
+1 530-210-6378
http://bixolabs.com
e l a s t i c   w e b   m i n i n g






Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message