hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hassan Khan <hassankhan...@gmail.com>
Subject Re: Upgrading from Httpclient 3.1 to 4.5 - localhost:443 not responding
Date Thu, 04 May 2017 14:41:16 GMT
Hi,

I had one quick question .... The documentation at
https://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/io/SessionInputBuffer.html
says that

*Session input buffer for blocking connections*. This interface is similar
to InputStream class, but it also provides methods for reading lines of
text.

My 443 connector is a non-blocking one... how can I fix this issue....
*Htppclient
has a input buffer for non-blocking connections ?*

My tomcat 8.5 connector definition is :
<!-- Define a SSL HTTP/1.1 Connector on port 443
         Uncomment the section below for a locally installed
connectionTimeout="-1"
         Novo Grid and use of a self signed certificate is required
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
         for communication to a locally installed Rendezvous -->

        <Connector port="443"
protocol="org.apache.coyote.http11.Http11NioProtocol"
====> Non-blocking connector
        maxThreads="200" SSLEnabled="true" compression="on"
minSpareThreads="25" maxSpareThreads="75"
        enableLookups="false"    connectionTimeout="-1"
        scheme="https" secure="true">

            <SSLHostConfig honorCipherOrder="false" >
                <Certificate certificateKeystoreFile="XXX"
                certificateKeystorePassword="XXX"
                certificateKeyAlias="localhost"
                type="RSA" />
            </SSLHostConfig>
        </Connector>

Any inputs?

Thanks
Hassan


On Wed, May 3, 2017 at 5:36 PM, Hassan Khan <hassankhan986@gmail.com> wrote:

> Thanks for the tip....
> After further debugging ... have a usual solution... the code work great
> with apache 2.4 ... but not with tomcat 8.5
> The changes needed were looks like the http 4.5 SSL validation is more
> stricter.
> With 3.1 ===> our certificate was applicable for *.Domain, our server
> using it was names X.V.Domain == it worked...
> With 4.5 ===> the SSL connection gave us a error so we cut our server name
> to V.Domian and it worked...
>
> But the most interesting thing that I cannot understand during stepping
> throught he code in eclipse the problem was noted in the code below in
> DefaultHttpResponseParser.java file for the function (Line 138) :
>
> protected HttpResponse parseHead(
>             final SessionInputBuffer sessionBuffer) throws IOException,
> HttpException {
>         //read out the HTTP status string
>         int count = 0;
>         ParserCursor cursor = null;
>         do {
>             // clear the buffer
>             this.lineBuf.clear();
>             final int i = sessionBuffer.readLine(this.lineBuf);  //====>
> whys is this empty for tomcat 8.5 sometimes and not for apache server 2.4
>             if (i == -1 && count == 0) {
>                 // The server just dropped connection on us
>                 throw new NoHttpResponseException("The target server
> failed to respond");
>             }
>             cursor = new ParserCursor(0, this.lineBuf.length());
>
> So kinda looks like there is socket level setting i am missing....
> Note : evey 20 secs we are pinging a servlet to get data from the server...
>
> Any ideas let me know.
>
> Thanks
> Hassan
>
> On Wed, Apr 26, 2017 at 6:53 PM, Bernd Eckenfels <ecki@zusammenkunft.net>
> wrote:
>
>> BTW: If you serialized your object into a BOS you catcall toByteArray()
>> and give that as an in memory entity content, there is no point to put an
>> input stream on top of it to feed the entity.
>>
>> Gruss
>> Bernd
>> --
>> http://bernd.eckenfels.net
>> ________________________________
>> From: Hassan Khan <hassankhan986@gmail.com>
>> Sent: Wednesday, April 26, 2017 10:17:44 PM
>> To: HttpClient User Discussion
>> Subject: Re: Upgrading from Httpclient 3.1 to 4.5 - localhost:443 not
>> responding
>>
>> Hi ,
>>
>> Wanted to know if some one can answer top of their head if
>> OutputObjectStreams are supported in 4.5 (3.0 does).
>> All our objects are marked as serializable.
>> we are using it like below to send a ObjectOutputStream to the post
>> method:
>>
>> ByteArrayOutputStream baos = new ByteArrayOutputStream();
>> oos = new ObjectOutputStream(baos);
>>  if (obj != null) {
>>      oos.writeObject(obj);
>>    }
>>    // create an input stream out of the stream
>>    // we put objects into
>>    InputStreamEntity temp = new InputStreamEntity(new
>> ByteArrayInputStream(baos.toByteArray()));
>>    postMethod.setEntity(temp);
>>
>>
>> Also while receiving the response,  is the method to get response as a
>> stream changed from
>>
>> 3.0 ==> ObjectInputStream instream = postMethod.getResponseBodyAsStream()
>>
>> to
>>
>> 4.5 ==> ObjectInputStream instream = new
>> ObjectInputStream(entity.getContent())  ?
>>
>> Thanks
>> Hassan
>>
>> On Tue, Apr 25, 2017 at 3:45 PM, Hassan Khan <hassankhan986@gmail.com>
>> wrote:
>>
>> > Hi All,
>> >
>> >   I have been trying to solve the below issue that show up in logs and
>> > unable to solve it from 2 weeks:
>> >
>> > 04/25/2017 15:37:30:513    FINEST: Invoking post method:
>> > https://localhost/localTomcat8_Rendezvous/comm/sendPayload
>> [Thread-23:
>> > Downloader for default channel (C:\Program Files\Apache Software
>> > Foundation\Tomcat 8.5\Instance1\webapps\localTom
>> cat8_Rendezvous\common),
>> > com.novoInnovations.network.protocol.Connection:sendRequest]
>> >
>> > 04/25/2017 15:22:33:677    WARNING: Could not get payload from
>> > localTomcat8_Rendezvous located at the path
>> localhost/localTomcat8_Rendezvous/comm
>> > [Thread-23: Downloader for default channel (C:\Program Files\Apache
>> > Software Foundation\Tomcat 8.5\Instance1\webapps\localTom
>> cat8_Rendezvous\common),
>> > com.novoInnovations.network.protocol.N2RProtocol:recvPayload]
>> >     localhost:443 failed to respond
>> >     org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(
>> > DefaultHttpResponseParser.java:143)
>> >     org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(
>> > DefaultHttpResponseParser.java:57)
>> >     org.apache.http.impl.io.AbstractMessageParser.parse(
>> > AbstractMessageParser.java:259)
>> >     org.apache.http.impl.DefaultBHttpClientConnection.
>> > receiveResponseHeader(DefaultBHttpClientConnection.java:163)
>> >     org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(
>> > CPoolProxy.java:167)
>> >     org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(
>> > HttpRequestExecutor.java:273)
>> >     org.apache.http.protocol.HttpRequestExecutor.execute(
>> > HttpRequestExecutor.java:125)
>> >     org.apache.http.impl.execchain.MainClientExec.
>> > execute(MainClientExec.java:271)
>> >     org.apache.http.impl.execchain.ProtocolExec.
>> > execute(ProtocolExec.java:184)
>> >     org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
>> >     org.apache.http.impl.execchain.RedirectExec.
>> > execute(RedirectExec.java:110)
>> >     org.apache.http.impl.client.InternalHttpClient.doExecute(
>> > InternalHttpClient.java:184)
>> >     org.apache.http.impl.client.CloseableHttpClient.execute(
>> > CloseableHttpClient.java:82)
>> >     org.apache.http.impl.client.CloseableHttpClient.execute(
>> > CloseableHttpClient.java:107)
>> >     com.novoInnovations.network.protocol.Connection.
>> > sendRequest(Connection.java:329)
>> >     com.novoInnovations.network.protocol.N2RProtocol.
>> > recvPayload(N2RProtocol.java:137)
>> >     com.novoInnovations.network.node.Downloader.
>> > getFromRendezvous(Downloader.java:411)
>> >     com.novoInnovations.network.node.Downloader.run(Downloader.
>> java:141)
>> > the next failure is at 15:22:53:743 , 15:23:13:853, .. so on
>> >
>> > The code is below (commented out are the many other ways that i tried to
>> > solve the issue...), this code is called every 20 secs by a thread, the
>> > target is a url and 443 port since it is https , it works the first
>> time ..
>> > the post methid has this url set :https://localhost/
>> > localTomcat8_Rendezvous/comm/sendPayload as
>> > postMethod = new HttpPost(this.url);
>> >
>> > Main function :
>> > public String sendRequest(Object obj) throws Exception {
>> >         // temporary output stream to stream objects into
>> >         ByteArrayOutputStream baos = new ByteArrayOutputStream();
>> >         oos = new ObjectOutputStream(baos);
>> >         if (isValid) {
>> >             myNovoLogger.finest("Writing node communicator version: " +
>> > VERSION);
>> >             oos.writeObject(VERSION);
>> >             myNovoLogger.finest("Writing node id: " + this.node.id());
>> >             oos.writeObject(this.node.id());
>> >             myNovoLogger.finest("Writing node's state");
>> >             oos.writeObject(this.state);
>> >             myNovoLogger.finest("Writing communication channel: " +
>> > this.getCommunicationChannel());
>> >             oos.writeObject(Integer.toString(this.
>> > getCommunicationChannel()));
>> >             if (obj != null) {
>> >                 myNovoLogger.finest("Streaming provided object.");
>> >                 oos.writeObject(obj);
>> >             }
>> >             // create an input stream out of the stream
>> >             // we put objects into
>> > /*            InputStream tis = new ByteArrayInputStream(baos.
>> > toByteArray());
>> >             /// take the created inputstream and make this the method's
>> > bidy
>> >             InputStreamEntity inputStreamEntity = new
>> > InputStreamEntity(tis);
>> >             HttpEntity httpEntity = new InputStreamEntity(tis,
>> > tis.available());
>> >             //ISSUE in uploading is from here
>> >             postMethod.setEntity(inputStreamEntity);*/
>> >
>> >             InputStreamEntity temp = new InputStreamEntity(new
>> > ByteArrayInputStream(baos.toByteArray()));
>> >             postMethod.setEntity(temp);
>> >
>> >
>> >
>> >             //postMethod.setRequestBody(tis);
>> >
>> >              RequestConfig config = RequestConfig.custom()
>> >                     .setConnectTimeout(20 * 1000)
>> >                     .setConnectionRequestTimeout(10* 60 * 1000)
>> >                     .setStaleConnectionCheckEnabled(true)
>> >                     .setSocketTimeout(10 * 60 * 1000).build();
>> >
>> >             myNovoLogger.finest("Invoking post method: " +
>> > postMethod.getURI().toString());
>> >             // execute method on server
>> >
>> >             PoolingHttpClientConnectionManager connManager = new
>> > PoolingHttpClientConnectionManager();
>> >             connManager.setValidateAfterInactivity(200);
>> >             ConnectionConfig connectionConfig =
>> ConnectionConfig.custom()
>> >           .setBufferSize(1000)
>> >           .build();
>> >            // HttpClientBuilder clientBuilder = HttpClients.custom();
>> >           //  clientBuilder.setRetryHandler(new
>> > DefaultHttpRequestRetryHandler(3, false));
>> >             // HttpClient client = clientBuilder.
>> > setDefaultConnectionConfig(connectionConfig).build();
>> >             //HttpHost targetHost = new HttpHost(url, 443, "https");
>> >             //CloseableHttpResponse response =
>> client1.execute(targetHost,
>> > postMethod);
>> >
>> >
>> >
>> >             this.response =  client.execute(postMethod);  //====> line
>> 329
>> >             String responseLine = null;
>> >
>> >             /* HttpHost targetHost = new HttpHost(this.url);
>> >              this.response = client.execute(targetHost, postMethod);*/
>> >             int rc =0;
>> >             try {
>> >             HttpEntity entity = response.getEntity();
>> >             if (entity != null) {
>> >                 entity = new BufferedHttpEntity(entity);
>> >             }
>> >             if (entity != null) {
>> >                 ObjectInputStream instream = new
>> ObjectInputStream(entity.
>> > getContent());
>> >                 try {
>> >                         myNovoLogger.finest("Getting response code.");
>> >                         myNovoLogger.finest("Got response:" +
>> > response.getStatusLine());
>> >                         rc = response.getStatusLine().getStatusCode();
>> >                         if (rc > 200) {
>> >                             myNovoLogger.warning("Response of " + rc);
>> >                             myNovoLogger.warning("ResponseBody:" +
>> > response.getStatusLine().getReasonPhrase());
>> >                         }
>> >                         myNovoLogger.finest("Getting input stream.");
>> >                         //ois = new ObjectInputStream(entity.
>> > getContent());
>> >                         //ois = new ObjectInputStream(postMethod.
>> > getResponseBodyAsStream());
>> >                         myNovoLogger.finest("Done sending request.");
>> >                         if(instream != null) {
>> >                             myNovoLogger.finest("responseLine
>> instream"+
>> > EntityUtils.toString(entity));
>> >                             responseLine = (String)
>> instream.readObject();
>> >                             setResponseLine(responseLine);
>> >                             if (Protocol.ACK.equals(responseLine)) {
>> >                             Object o = instream.readObject();
>> >                             setPayload((Payload) o);
>> >                             }
>> >                             Object o2 = instream.readObject();
>> >                             setDiscardList((HashMap) o2);
>> >                         //EntityUtils.consume(response.getEntity());
>> >                         }
>> >                 } finally {
>> >                     myNovoLogger.finest("closing instream");
>> >                     if(instream != null) {
>> >                         myNovoLogger.finest("closing instream");
>> >                     instream.close();
>> >                     myNovoLogger.finest("after closing instream");
>> >
>> >                     }
>> >                         this.client.close();
>> >                     isValid = false;
>> >                     connManager.close();
>> >                 }
>> >             }
>> >         } finally {
>> >             myNovoLogger.finest("Attempt to send request "+rc);
>> >
>> >             return responseLine;
>> >         }
>> >
>> > (Additional info : It works the first call then it fails
>> > consecutively...we are upgrading from client 3.2 to 4.5 , have included
>> > both core and client in the tomcat load path)
>> >
>> > Can some one find any problem , any inputs are welcome. I am new to
>> > httpclient coding.
>> >
>> > Thanks
>> > Hassan
>> >
>> > On Sat, Apr 22, 2017 at 9:35 PM, Hassan Khan <hassankhan986@gmail.com>
>> > wrote:
>> >
>> >> Hi Oleg,
>> >>
>> >> Thanks for answering ... very excited (and humbled)  to hear from
>> you...
>> >>
>> >> Thanks
>> >> Hassan
>> >>
>> >> On Sat, Apr 22, 2017 at 12:26 PM, Oleg Kalnichevski <olegk@apache.org>
>> >> wrote:
>> >>
>> >>> On Fri, 2017-04-21 at 10:21 -0400, Hassan Khan wrote:
>> >>> > Hi all,
>> >>> >
>> >>> >   We have pinpointed the issue and looks like the code that deals
>> >>> > with
>> >>> > streaming a object from the client to the server.
>> >>> >   With the old code (Httpclient 3.1) we would stream the object
>> >>> > through a
>> >>> > ObjectOutputStream and on the server side read it through a
>> >>> > ObjectInputStream.
>> >>> >   But looks like that workflow is not working in the new httpclient
>> >>> > 4.5.
>> >>> >   Is there any example or resource that specifically shows how
>> >>> > streaming
>> >>> > objects work ?
>> >>> >
>> >>>
>> >>> See this section of the tutorial
>> >>>
>> >>> http://hc.apache.org/httpcomponents-client-4.5.x/tutorial/
>> html/fundamen
>> >>> tals.html#d5e95
>> >>> <http://hc.apache.org/httpcomponents-client-4.5.x/tutorial/
>> html/fundamentals.html#d5e95>
>> >>>
>> >>>
>> >>> What you most likely want is SerializableEntity
>> >>>
>> >>> http://hc.apache.org/httpcomponents-core-4.4.x/httpcore/
>> apidocs/org/apa
>> >>> che/http/entity/SerializableEntity.html
>> >>> <http://hc.apache.org/httpcomponents-core-4.4.x/httpcore/
>> apidocs/org/apache/http/entity/SerializableEntity.html>
>> >>>
>> >>> Oleg
>> >>>
>> >>>
>> >>> > Thanks
>> >>> > Hassan
>> >>> >
>> >>> > On Tue, Apr 18, 2017 at 9:27 AM, Hassan Khan <
>> hassankhan986@gmail.com
>> >>> > >
>> >>> > wrote:
>> >>> >
>> >>> > > Hi ,
>> >>> > >
>> >>> > > Sorry the log images was filtered out. you can look at the
>> uploaded
>> >>> > > image.
>> >>> > >
>> >>> > > http://imgur.com/a/Nxpcw
>> >>> > >
>> >>> > > in text the logs say: I/O read timed out.
>> >>> > >
>> >>> > > Thanks
>> >>> > > Hassan
>> >>> > >
>> >>> > > On Mon, Apr 17, 2017 at 1:35 PM, Gary Gregory
>> <garydgregory@gmail.c
>> >>> > > om>
>> >>> > > wrote:
>> >>> > >
>> >>> > > > Hassan,
>> >>> > > >
>> >>> > > > Your attachments were filtered out. You might want to
try an
>> >>> > > > image sharing
>> >>> > > > site.
>> >>> > > >
>> >>> > > > Gary
>> >>> > > >
>> >>> > > > On Mon, Apr 17, 2017 at 6:16 AM, Hassan Khan
>> <hassankhan986@gmail
>> >>> > > > .com>
>> >>> > > > wrote:
>> >>> > > >
>> >>> > > > > +
>> >>> > > > > >
>> >>> > > > >
>> >>> > > > >
>> >>> > > > > > Hi All,
>> >>> > > > > >
>> >>> > > > > >
>> >>> > > > > >
>> >>> > > > > >    We are upgrading the httpclient in our software
from 3.1
>> >>> > > > > > to 4.5 (we
>> >>> > > > > > are adding both core and client). But we are
having some
>> >>> > > > > > issues in the
>> >>> > > > > > client and server communications. We are using
the below
>> >>> > > > > > client code
>> >>> > > > > > (simplified the code ) to make a  call every
60 secs and we
>> >>> > > > > > are getting
>> >>> > > > > > localhost not responding after few tries.
>> >>> > > > > >
>> >>> > > > > > Client code simplified is :
>> >>> > > > > >
>> >>> > > > > > RequestConfig config = RequestConfig.*custom*()
>> >>> > > > > >
>> >>> > > > > >               .setConnectTimeout(20 * 1000)
>> >>> > > > > >
>> >>> > > > > >               .setConnectionRequestTimeout(10*
60 * 1000)
>> >>> > > > > >
>> >>> > > > > >               .*setStaleConnectionCheckEnabled**(**true**)*
>> >>> > > > > >
>> >>> > > > > >               .setSocketTimeout(10 * 60 * 1000).build();
>> >>> > > > > >
>> >>> > > > > > *this*.client =
>> >>> > > > > > HttpClients.*custom*().setDefaultRequestConfig(config
>> >>> > > > > > ).build();
>> >>> > > > > >
>> >>> > > > > > HttpResponse response = client.execute(postMethod);
>> >>> > > > > >
>> >>> > > > > > *int* rc = response.getStatusLine().getStatusCode();
>> >>> > > > > >
>> >>> > > > > > *if* (rc > 200) {
>> >>> > > > > >
>> >>> > > > > >       log error
>> >>> > > > > >
>> >>> > > > > > }
>> >>> > > > > >
>> >>> > > > > > //using the object stream to read data..
>> >>> > > > > >
>> >>> > > > > > ois = *new*
>> >>> > > > > > ObjectInputStream(response.getEntity().getContent());
>> >>> > > > > >
>> >>> > > > > > //at the end we close it
>> >>> > > > > >
>> >>> > > > > > postMethod.releaseConnection();
>> >>> > > > > >
>> >>> > > > > >
>> >>> > > > > >
>> >>> > > > > > But still we are getting the following issue:
>> >>> > > > > >
>> >>> > > > > >
>> >>> > > > > > [image: cid:image003.png@01D2B446.A78B7290]
>> >>> > > > > >
>> >>> > > > > >
>> >>> > > > > >
>> >>> > > > > > Any recommendations would be appreciated. we
have 2 thread
>> >>> > > > > > making a
>> >>> > > > > > request every 60 secs. When we used a spooling
manager
>> always
>> >>> > > > > > one
>> >>> > > >
>> >>> > > > route was
>> >>> > > > > > used..
>> >>> > > > > >
>> >>> > > > >
>> >>> > > > >     May be both thread that are calling the same
URL
>> >>> > > > > simultaneously are
>> >>> > > > > sharing one connection , but they should share different
>> >>> > > > > connections..
>> >>> > > > >
>> >>> > > > > >
>> >>> > > > > >
>> >>> > > > > > Thanks
>> >>> > > > > >
>> >>> > > > > >
>> >>> > > > > >
>> >>> > > > > > *Hassan Khan*
>> >>> > > > > >
>> >>> > > > > > Software Developer
>> >>> > > > > >
>> >>> > > > > >
>> >>> > > > >
>> >>> > > > >
>> >>> > > > >
>> >>> > > >
>> >>> > > >
>> >>> > > > --
>> >>> > > > E-Mail: garydgregory@gmail.com | ggregory@apache.org
>> >>> > > > Java Persistence with Hibernate, Second Edition
>> >>> > > > <https://www.amazon.com/gp/product/1617290459/ref=as_li_tl?
>> >>> > > > ie=UTF8&camp=1789&creative=9325&creativeASIN=1617290459&link
>> >>> > > > Code=as2&tag=garygregory-
>> >>> > > > 20&linkId=cadb800f39946ec62ea2b1af9fe6a2b8>
>> >>> > > >
>> >>> > > > <http:////ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=
>> >>> > > > am2&o=1&a=1617290459>
>> >>> > > > JUnit in Action, Second Edition
>> >>> > > > <https://www.amazon.com/gp/product/1935182021/ref=as_li_tl?
>> >>> > > > ie=UTF8&camp=1789&creative=9325&creativeASIN=1935182021&link
>> >>> > > > Code=as2&tag=garygregory-
>> >>> > > > 20&linkId=31ecd1f6b6d1eaf8886ac902a24de418%22>
>> >>> > > >
>> >>> > > > <http:////ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=
>> >>> > > > am2&o=1&a=1935182021>
>> >>> > > > Spring Batch in Action
>> >>> > > > <https://www.amazon.com/gp/product/1935182951/ref=as_li_tl?
>> >>> > > > ie=UTF8&camp=1789&creative=9325&creativeASIN=1935182951&link
>> >>> > > > Code=%7B%7BlinkCode%7D%7D&tag=garygregory-20&linkId=%7B%7Bli
>> >>> > > > nk_id%7D%7D%22%3ESpring+Batch+in+Action>
>> >>> > > > <http:////ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=
>> >>> > > > am2&o=1&a=1935182951>
>> >>> > > > Blog: http://garygregory.wordpress.com
>> >>> > > > Home: http://garygregory.com/
>> >>> > > > Tweet! http://twitter.com/GaryGregory
>> >>> > > >
>> >>> > >
>> >>> > >
>> >>> > >
>> >>> > > --
>> >>> > > Hassan Khan
>> >>> > >
>> >>> >
>> >>> >
>> >>> >
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
>> >>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> Hassan Khan
>> >>
>> >
>> >
>> >
>> > --
>> > Hassan Khan
>> >
>>
>>
>>
>> --
>> Hassan Khan
>>
>
>
>
> --
> Hassan Khan
>



-- 
Hassan Khan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message