hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kwik, Micky" <micky.k...@sogeti.nl>
Subject HttpClient returns different response compared to browser
Date Tue, 20 May 2008 12:55:07 GMT
Hi,
 
I wrote a simple client to fetch documents from some websites. But I found that the HttpClient
often gets a different response compared to the browser even if the HTTP status code is 200.
For example this URL: http://www.elsevierfiscaal.nl/els/enc/productserviceoverzicht/id1101-31813/search/true/channelId/1101/update-14-aangifte-assistent-2008.html
<https://webmail.sogeti.nl/exchweb/bin/redir.asp?URL=http://www.elsevierfiscaal.nl/els/enc/productserviceoverzicht/id1101-31813/search/true/channelId/1101/update-14-aangifte-assistent-2008.html>
 or http://www.belastingdienst.nl/zakelijk/nieuwsbrief/nieuwsberichten/2008-04-02-08_franke.html
<https://webmail.sogeti.nl/exchweb/bin/redir.asp?URL=http://www.belastingdienst.nl/zakelijk/nieuwsbrief/nieuwsberichten/2008-04-02-08_franke.html>

 
Here is my code snippet:
  HttpClient client = setUpClient(aUrl);
  GetMethod method = new GetMethod();
  method.getParams().setParameter("http.useragent", "Mozilla/5.0 (Windows; U; Windows NT 5.1;
en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14");
  method.getParams().setBooleanParameter("http.protocol.single-cookie-header", true);
  method.setFollowRedirects(false);
  method.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY);
  
  int returnCode = client.executeMethod(method);
  if (returnCode == HttpStatus.SC_OK) {
   fetchBody(aUrl, method);
  }
I already varied Cookie policies and followRedirects but with no success. Is there a way of
solving this or is it a case of "http client is not a browser" ?
 
Kind regards,
Micky

Disclaimer:
This message contains information that may be privileged or confidential and is the property
of Sogeti Nederland B.V. or its Group members. It is intended only for the person to whom
it is addressed. If you are not the intended recipient, you are not authorized to read, print,
retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive
this message in error, please notify the sender immediately and delete all copies of this
message.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message