Mailing-List: contact dev-help@hc.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "HttpComponents Project" <dev@hc.apache.org>
Received-SPF: pass (athena.apache.org: domain of daddywri@gmail.com designates
 209.85.213.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CALUFAGAOa87+8=VpxfzLaXUnALkgwa9urfrAeWvAVrdw+n2Btw@mail.gmail.com>
References: 
 <CALUFAGAwhm=S_SPpg-K+mtQUw2tthkzfbcdJ46UYAH4mwRZNeQ@mail.gmail.com>
	<1400678661.5757.3.camel@ubuntu>
	<CALUFAGAOa87+8=VpxfzLaXUnALkgwa9urfrAeWvAVrdw+n2Btw@mail.gmail.com>
Date: Wed, 21 May 2014 10:09:26 -0400
Message-ID: 
 <CALUFAGDmiksF43S+yhR+1cEjRpqhWFnsgBBj0sAKSMPyUEs7pQ@mail.gmail.com>
Subject: Re: Expect-continue doesn't seem operative using 4.3.x builder
 structures
From: Karl Wright <daddywri@gmail.com>
To: HttpComponents Project <dev@hc.apache.org>
Cc: dev <dev@manifoldcf.apache.org>
Content-Type: multipart/alternative; boundary=20cf303dd74a50a58a04f9e98b12

--20cf303dd74a50a58a04f9e98b12
Content-Type: text/plain; charset=UTF-8

FWIW, if it helps you can look at the complete log here:

http://folk.uio.no/erlendfg/manifoldcf/manifoldcf.log

This includes GETs to other machines as well as POSTs to Solr.  It's the
POSTs though that are what is failing.

Karl


On Wed, May 21, 2014 at 10:01 AM, Karl Wright <daddywri@gmail.com> wrote:

> Hi Oleg,
>
> Here are the headers it sends in our case.  This is 4.3.3.
>
> >>>>>>
>
> DEBUG 2014-05-21 15:28:39,630 (Thread-449) - http-outgoing-6 >> POST /solr/uio/update?wt=xml&version=2.2 HTTP/1.1
> DEBUG 2014-05-21 15:28:39,630 (Thread-449) - http-outgoing-6 >> User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrServer] 1.0
> DEBUG 2014-05-21 15:28:39,630 (Thread-449) - http-outgoing-6 >> Transfer-Encoding: chunked
> DEBUG 2014-05-21 15:28:39,630 (Thread-456) - http-outgoing-5 >> POST /solr/uio/update/extract HTTP/1.1
> DEBUG 2014-05-21 15:28:39,631 (Thread-449) - http-outgoing-6 >> Content-Type: application/xml; charset=UTF-8
> DEBUG 2014-05-21 15:28:39,631 (Thread-456) - http-outgoing-5 >> Content-Charset: UTF-8
> DEBUG 2014-05-21 15:28:39,631 (Thread-449) - http-outgoing-6 >> Host: solr-test01.uio.no:443
> DEBUG 2014-05-21 15:28:39,631 (Thread-456) - http-outgoing-5 >> User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrServer] 1.0
> DEBUG 2014-05-21 15:28:39,631 (Thread-449) - http-outgoing-6 >> Connection: Keep-Alive
> DEBUG 2014-05-21 15:28:39,631 (Thread-456) - http-outgoing-5 >> Transfer-Encoding: chunked
> DEBUG 2014-05-21 15:28:39,631 (Thread-449) - http-outgoing-6 >> Accept-Encoding: gzip,deflate
> DEBUG 2014-05-21 15:28:39,631 (Thread-456) - http-outgoing-5 >> Content-Type: multipart/form-data; boundary=efUhz7rIZVmTta4q-godYmOjxfvqHlT3Ppmc; charset=UTF-8
> DEBUG 2014-05-21 15:28:39,631 (Thread-456) - http-outgoing-5 >> Host: solr-test01.uio.no:443
> DEBUG 2014-05-21 15:28:39,631 (Thread-456) - http-outgoing-5 >> Connection: Keep-Alive
> DEBUG 2014-05-21 15:28:39,631 (Thread-456) - http-outgoing-5 >> Accept-Encoding: gzip,deflate
> DEBUG 2014-05-21 15:28:39,632 (Thread-449) - http-outgoing-6 >> "POST /solr/uio/update?wt=xml&version=2.2 HTTP/1.1[\r][\n]"
> DEBUG 2014-05-21 15:28:39,632 (Thread-449) - http-outgoing-6 >> "User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrServer] 1.0[\r][\n]"
> DEBUG 2014-05-21 15:28:39,632 (Thread-449) - http-outgoing-6 >> "Transfer-Encoding: chunked[\r][\n]"
> DEBUG 2014-05-21 15:28:39,632 (Thread-449) - http-outgoing-6 >> "Content-Type: application/xml; charset=UTF-8[\r][\n]"
> DEBUG 2014-05-21 15:28:39,632 (Thread-449) - http-outgoing-6 >> "Host: solr-test01.uio.no:443[\r][\n]"
> DEBUG 2014-05-21 15:28:39,632 (Thread-449) - http-outgoing-6 >> "Connection: Keep-Alive[\r][\n]"
> DEBUG 2014-05-21 15:28:39,632 (Thread-449) - http-outgoing-6 >> "Accept-Encoding: gzip,deflate[\r][\n]"
> <<<<<<
>
> Expect-continue is *not* firing, as you can see.  Something else we are doing in the builder, or maybe that httpclient is doing, is apparently preventing it.
>
> Suggestions for how to quickly debug this problem?
>
> Karl
>
>
>
>
> On Wed, May 21, 2014 at 9:24 AM, Oleg Kalnichevski <olegk@apache.org>wrote:
>
>> On Wed, 2014-05-21 at 08:52 -0400, Karl Wright wrote:
>> > Hi Oleg,
>> >
>> > A few months back, if you will recall, I ported much of ManifoldCF to
>> use
>> > the HttpClient 4.3 builder structures for its various connectors.
>> >
>> > Unfortunately, one thing we'd previously fixed came unfixed when I did
>> > this.  We have a user who has a site that is being posted to that
>> requires
>> > basic auth.  We were getting an exception with HttpClient 4.2 having to
>> do
>> > with non-repeatable request entities.  We solved that by turning on
>> > expect-continue.
>> >
>> > For 4.3, we still have expect-continue on, but now we are getting
>> similar
>> > problems:
>> >
>> > >>>>>>
>> >
>> > Caused by: org.apache.http.client.NonRepeatableRequestException:
>> > Cannot retry request with a non-repeatable request entity.
>> >       at
>> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:208)
>> >       at
>> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
>> >       at
>> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
>> >       at
>> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
>> > >>>>>>
>> >
>> > The code used to set up HttpClient looks like this:
>> >
>> > >>>>>>
>> >     RequestConfig.Builder requestBuilder = RequestConfig.custom()
>> >       .setCircularRedirectsAllowed(true)
>> >       .setSocketTimeout(socketTimeout)
>> >       .setStaleConnectionCheckEnabled(true)
>> >       .setExpectContinueEnabled(true)
>> >       .setConnectTimeout(connectionTimeout)
>> >       .setConnectionRequestTimeout(socketTimeout);
>> >
>> >       HttpClientBuilder clientBuilder = HttpClients.custom()
>> >         .setConnectionManager(connectionManager)
>> >         .setMaxConnTotal(1)
>> >         .disableAutomaticRetries()
>> >         .setDefaultRequestConfig(requestBuilder.build())
>> >         .setRedirectStrategy(new DefaultRedirectStrategy())
>> >         .setSSLSocketFactory(myFactory)
>> >         .setRequestExecutor(new HttpRequestExecutor(socketTimeout))
>> >         .setDefaultSocketConfig(SocketConfig.custom()
>> >           .setTcpNoDelay(true)
>> >           .setSoTimeout(socketTimeout)
>> >           .build()
>> >         );
>> >
>> >
>> >     if (userID != null && userID.length() > 0 && password != null)
>> >     {
>> >       CredentialsProvider credentialsProvider = new
>> BasicCredentialsProvider();
>> >       Credentials credentials = new
>> > UsernamePasswordCredentials(userID, password);
>> >       if (realm != null)
>> >         credentialsProvider.setCredentials(new
>> > AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, realm),
>> > credentials);
>> >       else
>> >         credentialsProvider.setCredentials(AuthScope.ANY, credentials);
>> >
>> >       clientBuilder.setDefaultCredentialsProvider(credentialsProvider);
>> >     }
>> >
>> >     HttpClient localClient = clientBuilder.build();
>> > <<<<<<
>> >
>> > Wire logs show *no* expect/continue at all taking place.  Can you tell
>> > me what we are doing wrong?
>> >
>> > Thanks,
>> > Karl
>>
>> I see nothing wrong with your code. At the same time I executed the test
>> app below and got the expected behavior with regards to the
>> expect-continue handshaking.
>>
>> ---
>> RequestConfig requestConfig = RequestConfig.custom()
>>         .setExpectContinueEnabled(true)
>>         .build();
>> CloseableHttpClient client = HttpClients.custom()
>>         .setDefaultRequestConfig(requestConfig)
>>         .build();
>> HttpPost post = new HttpPost("http://www.google.ch/");
>> post.setEntity(new StringEntity("stuff", ContentType.TEXT_PLAIN));
>> CloseableHttpResponse response = client.execute(post);
>> try {
>>
>> } finally {
>>     response.close();
>> }
>> ---
>>
>> ---
>> [DEBUG] RequestAddCookies - CookieSpec selected: best-match
>> [DEBUG] RequestAuthCache - Auth cache not set in the context
>> [DEBUG] PoolingHttpClientConnectionManager - Connection request: [route:
>> {}->http://www.google.ch:80][total kept alive: 0; route allocated: 0 of
>> 2; total allocated: 0 of 20]
>> [DEBUG] PoolingHttpClientConnectionManager - Connection leased: [id:
>> 0][route: {}->http://www.google.ch:80][total kept alive: 0; route
>> allocated: 1 of 2; total allocated: 1 of 20]
>> [DEBUG] MainClientExec - Opening connection {}->http://www.google.ch:80
>> [DEBUG] HttpClientConnectionOperator - Connecting to
>> www.google.ch/173.194.113.152:80
>> [DEBUG] HttpClientConnectionOperator - Connection established
>> 192.168.42.101:39266<->173.194.113.152:80
>> [DEBUG] MainClientExec - Executing request POST / HTTP/1.1
>> [DEBUG] MainClientExec - Target auth state: UNCHALLENGED
>> [DEBUG] MainClientExec - Proxy auth state: UNCHALLENGED
>> [DEBUG] headers - http-outgoing-0 >> POST / HTTP/1.1
>> [DEBUG] headers - http-outgoing-0 >> Content-Length: 5
>> [DEBUG] headers - http-outgoing-0 >> Content-Type: text/plain;
>> charset=ISO-8859-1
>> [DEBUG] headers - http-outgoing-0 >> Host: www.google.ch
>> [DEBUG] headers - http-outgoing-0 >> Connection: Keep-Alive
>> [DEBUG] headers - http-outgoing-0 >> User-Agent: Apache-HttpClient/4.3.3
>> (java 1.5)
>> [DEBUG] headers - http-outgoing-0 >> Expect: 100-continue
>> [DEBUG] headers - http-outgoing-0 >> Accept-Encoding: gzip,deflate
>> [DEBUG] headers - http-outgoing-0 << HTTP/1.1 100 Continue
>> [DEBUG] headers - http-outgoing-0 << HTTP/1.1 405 Method Not Allowed
>> [DEBUG] headers - http-outgoing-0 << Content-Type: text/html;
>> charset=UTF-8
>> [DEBUG] headers - http-outgoing-0 << Content-Length: 1453
>> [DEBUG] headers - http-outgoing-0 << Allow: GET, HEAD
>> [DEBUG] headers - http-outgoing-0 << Alternate-Protocol: 80:quic
>> [DEBUG] headers - http-outgoing-0 << Date: Wed, 21 May 2014 13:19:01 GMT
>> [DEBUG] headers - http-outgoing-0 << Server: gws
>> [DEBUG] headers - http-outgoing-0 << X-Frame-Options: SAMEORIGIN
>> [DEBUG] headers - http-outgoing-0 << X-XSS-Protection: 1; mode=block
>> [DEBUG] MainClientExec - Connection can be kept alive indefinitely
>> [DEBUG] DefaultManagedHttpClientConnection - http-outgoing-0: Shutdown
>> connection
>> [DEBUG] MainClientExec - Connection discarded
>> [DEBUG] DefaultManagedHttpClientConnection - http-outgoing-0: Close
>> connection
>> [DEBUG] PoolingHttpClientConnectionManager - Connection released: [id:
>> 0][route: {}->http://www.google.ch:80][total kept alive: 0; route
>> allocated: 0 of 2; total allocated: 0 of 20]
>> ---
>>
>> Oleg
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>> For additional commands, e-mail: dev-help@hc.apache.org
>>
>>
>

--20cf303dd74a50a58a04f9e98b12--