hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From terry_513 <trupt...@hotmail.com>
Subject Re: Why is POST (HttpClient) is acting like this?
Date Wed, 21 Nov 2007 14:49:00 GMT



Its a desktop application. I have set Full wire + context Log. At present,
the post response comes on the same page.

[code]
           client = new HttpClient();
 //        client.getParams().setParameter("http.protocol.version",
HttpVersion.HTTP_1_1);
         client.getHostConfiguration().setHost(ymailSinupUrl, 443, "https");
         connectMgr = client.getHttpConnectionManager();
         hc = client.getHostConfiguration();
         hc.getParams().setParameter("http.protocol.version",
HttpVersion.HTTP_1_0);
         client.getParams().setCookiePolicy(null);
........
  ......
       NameValuePair data[] = {
            new NameValuePair("u", _dsh), new NameValuePair("t", _t1), 
            new NameValuePair("preferredcontent", this.preferLang), new
NameValuePair("firstname", FirstName), new NameValuePair("secondname",
LastName),  new NameValuePair("gender", this.gender), new
NameValuePair("mm", this.bMm), new NameValuePair("dd", bDate), new
NameValuePair("yyyy", bYyyy), new NameValuePair("country", loc), new
NameValuePair("postalcode", postalCode), 
            new NameValuePair("yahooid", Email), new
NameValuePair("password", Passwrd), new NameValuePair("passwordconfirm",
passwrdAgain), 
            new NameValuePair("altemail", this.alterEmail), new
NameValuePair("secquestion", selection), new
NameValuePair("secquestionanswer", this.IdentifyAnswer), new
NameValuePair("cword", newaccountcaptcha), new NameValuePair("cdata",
this._continue),  new NameValuePair("tos_agreed", this.doAgree), new
NameValuePair("IAgreeBtn", submitbutton), new NameValuePair("action",
formAction)
        };

...........

    public String POST(String url, NameValuePair[] data) {
        String res = "";
        String URL = ymailSinupUrl + "&_ylt=A9FJpMBjCkRHERcBDgCZ2PAI";
        PostMethod method = new PostMethod(URL);
        // SET PROPERTIES 
        method.getParams().setParameter("http.method.retry-handler", new
DefaultHttpMethodRetryHandler(3, false));
        method.setRequestHeader("User-agent", USER_AGENT);
        method.setRequestHeader("Accept", ACCEPT);
        method.setRequestHeader("Accept-Language", ACCEPT_LANG);
        method.setRequestHeader("Accept-Charset", ACCEPT_CHAR);
        method.setRequestHeader("Keep-Alive", KEEP_ALIVE);
        method.setRequestHeader("Connection", CONNECTION);
        method.setRequestHeader("Content-Type", "text/html");
        
 
        if (! "".equals(lastUrl))
            method.setRequestHeader("Referer", lastUrl);
        
        if (connectMgr.getConnection(hc).isOpen() == true)
            System.out.println("Connection is OPEN");
        else {
            System.out.println("Connection is ******************* NOT OPEN
*************** ");
        }
        method.setRequestBody(data);
 
        int statusCode = 0;
        try {
            statusCode = client.executeMethod(hc, method);
            System.out.println("Register Send: " +
method.getStatusLine().toString());
        }catch (HttpException e) {
            method.releaseConnection();
            System.out.println("HTTP EXception : " + e.getMessage());
        }catch (IOException ie) {
            method.releaseConnection();
            System.out.println("Error Exe Method - Post. Status Code = " +
statusCode);
            ie.printStackTrace();
        }
        
        if (statusCode != 200) {
            System.err.println((new StringBuilder()).append("POST Method
failed: ").append(method.getStatusLine()).toString());
        } // else {
            InputStream inputStream = null;
            BufferedReader input = null;
            try {
                inputStream = method.getResponseBodyAsStream();
                input = new BufferedReader(new
InputStreamReader(inputStream));
                String str;
                while((str = input.readLine()) != null) {
                    res = (new
StringBuilder()).append(res).append(str).toString();
                }
                input.close();
            } catch (IOException ie) {
                method.releaseConnection();
                ie.printStackTrace();
            }
//        }
  
        try {
            lastUrl = method.getURI().toString();
        } catch (HttpException he) {
            he.printStackTrace();
        }finally {
            method.releaseConnection();
        }
        
        
        System.out.println((new StringBuilder()).append("Status Code =
").append(statusCode).toString());
        Header h;
        if (statusCode == 302) {
            h = method.getResponseHeader("Location");
            System.out.println("Header = " + h.getValue());
            
            // REQUIRES RE-DIRECT
            System.out.println("RE DIRECTING TARGET .....");
            res = GET(h.getValue());
        }
        return res;
    }

[/code]

in NameValuePairs, 
tos_agreed is a checkbox & I have assigned it 1. IAgreeBtn, has the same
value as the html source has - "Create My Account", action has the
formAction url. Other all parameters r proper, captcha is also entered
correctly. u, t r parsed from html source & taken dynamically & passed. I
had also added these parameters, but removed it as they remain same in all
pages.

//new NameValuePair("dracs", dracs), new NameValuePair("done", _done), new
NameValuePair("last", ""), new NameValuePair("partner", _partner), new
NameValuePair("src", new String()), new NameValuePair("intl", new
String("us")), new NameValuePair("jsenabled", new String("0")),
//, new NameValuePair("action", formAction)

// URL -> https://edit.yahoo.com/registration?.intl=us&new=1&.done=http
// formAction -> /registration;_ylt=A9FJpMTzzUJHpgIADgCZ2PAI
"POST /registration?.intl=us&new=1&.done=http&_ylt=A9FJpMBjCkRHERcBDgCZ2PAI
HTTP/1.0[\r][\n]"


This is the source of the post method & its required resources. Can anyone
know, where & what is the error? Why does the post refer to the same page
instead of next page. I am in very bad shape to solve this problem. Please
if, any one can help me out. Any help is highly appreciated.


Thanks 



sebb-2-2 wrote:
> 
> You could use a protocol analyser such as WireShark
> (www.wireshark.org) to compare the requests sent by a browser and
> those sent by your application.
> 
> Some browsers also offer add-ons to show the HTTP traffic. Check what
> the browser is sending, and ensure that your application follows that
> as closely as possible.
> 
> On 21/11/2007, terry_513 <truptikm@hotmail.com> wrote:
>>
>>
>> Thanks Oleg. After some Rnd, now the same page is coming on post
>> response.
>> Their are programs that work through such sites via desktop application.
>> What am I lacking, that I get wrong results? Their must be something to
>> work
>> with these sites also. Any idea or suggestion would be of great help. If
>> you
>> want, I can pass you the code.
>>
>> Please help me. I see in this site, you and other experts have helped so
>> many people. Please.
>>
>> Thanks
>>
>>
>>
>>
>> olegk wrote:
>> >
>> >
>> > On Tue, 2007-11-20 at 23:15 -0800, terry_513 wrote:
>> >>
>> >> Hello,
>> >>
>> >>   In my desktop application, using HttpClient 3.1, I am navigating a
>> web
>> >> site. I am trying to retreive the site
>> >> https://edit.yahoo.com/registration?.intl=us&new=1&.done=http, I
pass
>> the
>> >> respective parameters and perform a POST. The post requires redirect.
>> So,
>> >> I
>> >> get the "Location" from the header which points to "http://yahoo.com"
>> and
>> >> get the new location url via GET method.
>> >>
>> >>    My problem is: the submission if form should be going somehwere
>> else,
>> >> but
>> >> it is going to the home page. Why so? Where I may be going wrong? The
>> >> code
>> >> doesn't throw any exception or error. But the page that I receive is
>> not
>> >> expected. Any idea, guidance will be of great help.
>> >>
>> >> Thanks
>> >>
>> >>
>> >
>> >
>> > This has nothing to do with HttpClient. Yahoo and many other high
>> > profile sites are known to employ various techniques to detect and
>> > prevent automated screen-scraping.
>> >
>> > Oleg
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
>> > For additional commands, e-mail:
>> httpclient-user-help@jakarta.apache.org
>> >
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Why-is-POST-%28HttpClient%29-is-acting-like-this--tf4848502.html#a13875050
>> Sent from the HttpClient-User mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-user-help@jakarta.apache.org
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Why-is-POST-%28HttpClient%29-is-acting-like-this--tf4848502.html#a13878617
Sent from the HttpClient-User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-user-help@jakarta.apache.org


Mime
View raw message