hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hassan Khan <hassankhan...@gmail.com>
Subject Re: Upgrading from Httpclient 3.1 to 4.5 - localhost:443 not responding
Date Wed, 03 May 2017 21:36:17 GMT
Thanks for the tip....
After further debugging ... have a usual solution... the code work great
with apache 2.4 ... but not with tomcat 8.5
The changes needed were looks like the http 4.5 SSL validation is more
stricter.
With 3.1 ===> our certificate was applicable for *.Domain, our server using
it was names X.V.Domain == it worked...
With 4.5 ===> the SSL connection gave us a error so we cut our server name
to V.Domian and it worked...

But the most interesting thing that I cannot understand during stepping
throught he code in eclipse the problem was noted in the code below in
DefaultHttpResponseParser.java file for the function (Line 138) :

protected HttpResponse parseHead(
            final SessionInputBuffer sessionBuffer) throws IOException,
HttpException {
        //read out the HTTP status string
        int count = 0;
        ParserCursor cursor = null;
        do {
            // clear the buffer
            this.lineBuf.clear();
            final int i = sessionBuffer.readLine(this.lineBuf);  //====>
whys is this empty for tomcat 8.5 sometimes and not for apache server 2.4
            if (i == -1 && count == 0) {
                // The server just dropped connection on us
                throw new NoHttpResponseException("The target server failed
to respond");
            }
            cursor = new ParserCursor(0, this.lineBuf.length());

So kinda looks like there is socket level setting i am missing....
Note : evey 20 secs we are pinging a servlet to get data from the server...

Any ideas let me know.

Thanks
Hassan

On Wed, Apr 26, 2017 at 6:53 PM, Bernd Eckenfels <ecki@zusammenkunft.net>
wrote:

> BTW: If you serialized your object into a BOS you catcall toByteArray()
> and give that as an in memory entity content, there is no point to put an
> input stream on top of it to feed the entity.
>
> Gruss
> Bernd
> --
> http://bernd.eckenfels.net
> ________________________________
> From: Hassan Khan <hassankhan986@gmail.com>
> Sent: Wednesday, April 26, 2017 10:17:44 PM
> To: HttpClient User Discussion
> Subject: Re: Upgrading from Httpclient 3.1 to 4.5 - localhost:443 not
> responding
>
> Hi ,
>
> Wanted to know if some one can answer top of their head if
> OutputObjectStreams are supported in 4.5 (3.0 does).
> All our objects are marked as serializable.
> we are using it like below to send a ObjectOutputStream to the post method:
>
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> oos = new ObjectOutputStream(baos);
>  if (obj != null) {
>      oos.writeObject(obj);
>    }
>    // create an input stream out of the stream
>    // we put objects into
>    InputStreamEntity temp = new InputStreamEntity(new
> ByteArrayInputStream(baos.toByteArray()));
>    postMethod.setEntity(temp);
>
>
> Also while receiving the response,  is the method to get response as a
> stream changed from
>
> 3.0 ==> ObjectInputStream instream = postMethod.getResponseBodyAsStream()
>
> to
>
> 4.5 ==> ObjectInputStream instream = new
> ObjectInputStream(entity.getContent())  ?
>
> Thanks
> Hassan
>
> On Tue, Apr 25, 2017 at 3:45 PM, Hassan Khan <hassankhan986@gmail.com>
> wrote:
>
> > Hi All,
> >
> >   I have been trying to solve the below issue that show up in logs and
> > unable to solve it from 2 weeks:
> >
> > 04/25/2017 15:37:30:513    FINEST: Invoking post method:
> > https://localhost/localTomcat8_Rendezvous/comm/sendPayload
> [Thread-23:
> > Downloader for default channel (C:\Program Files\Apache Software
> > Foundation\Tomcat 8.5\Instance1\webapps\localTomcat8_Rendezvous\common),
> > com.novoInnovations.network.protocol.Connection:sendRequest]
> >
> > 04/25/2017 15:22:33:677    WARNING: Could not get payload from
> > localTomcat8_Rendezvous located at the path localhost/localTomcat8_
> Rendezvous/comm
> > [Thread-23: Downloader for default channel (C:\Program Files\Apache
> > Software Foundation\Tomcat 8.5\Instance1\webapps\
> localTomcat8_Rendezvous\common),
> > com.novoInnovations.network.protocol.N2RProtocol:recvPayload]
> >     localhost:443 failed to respond
> >     org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(
> > DefaultHttpResponseParser.java:143)
> >     org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(
> > DefaultHttpResponseParser.java:57)
> >     org.apache.http.impl.io.AbstractMessageParser.parse(
> > AbstractMessageParser.java:259)
> >     org.apache.http.impl.DefaultBHttpClientConnection.
> > receiveResponseHeader(DefaultBHttpClientConnection.java:163)
> >     org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(
> > CPoolProxy.java:167)
> >     org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(
> > HttpRequestExecutor.java:273)
> >     org.apache.http.protocol.HttpRequestExecutor.execute(
> > HttpRequestExecutor.java:125)
> >     org.apache.http.impl.execchain.MainClientExec.
> > execute(MainClientExec.java:271)
> >     org.apache.http.impl.execchain.ProtocolExec.
> > execute(ProtocolExec.java:184)
> >     org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
> >     org.apache.http.impl.execchain.RedirectExec.
> > execute(RedirectExec.java:110)
> >     org.apache.http.impl.client.InternalHttpClient.doExecute(
> > InternalHttpClient.java:184)
> >     org.apache.http.impl.client.CloseableHttpClient.execute(
> > CloseableHttpClient.java:82)
> >     org.apache.http.impl.client.CloseableHttpClient.execute(
> > CloseableHttpClient.java:107)
> >     com.novoInnovations.network.protocol.Connection.
> > sendRequest(Connection.java:329)
> >     com.novoInnovations.network.protocol.N2RProtocol.
> > recvPayload(N2RProtocol.java:137)
> >     com.novoInnovations.network.node.Downloader.
> > getFromRendezvous(Downloader.java:411)
> >     com.novoInnovations.network.node.Downloader.run(Downloader.java:141)
> > the next failure is at 15:22:53:743 , 15:23:13:853, .. so on
> >
> > The code is below (commented out are the many other ways that i tried to
> > solve the issue...), this code is called every 20 secs by a thread, the
> > target is a url and 443 port since it is https , it works the first time
> ..
> > the post methid has this url set :https://localhost/
> > localTomcat8_Rendezvous/comm/sendPayload as
> > postMethod = new HttpPost(this.url);
> >
> > Main function :
> > public String sendRequest(Object obj) throws Exception {
> >         // temporary output stream to stream objects into
> >         ByteArrayOutputStream baos = new ByteArrayOutputStream();
> >         oos = new ObjectOutputStream(baos);
> >         if (isValid) {
> >             myNovoLogger.finest("Writing node communicator version: " +
> > VERSION);
> >             oos.writeObject(VERSION);
> >             myNovoLogger.finest("Writing node id: " + this.node.id());
> >             oos.writeObject(this.node.id());
> >             myNovoLogger.finest("Writing node's state");
> >             oos.writeObject(this.state);
> >             myNovoLogger.finest("Writing communication channel: " +
> > this.getCommunicationChannel());
> >             oos.writeObject(Integer.toString(this.
> > getCommunicationChannel()));
> >             if (obj != null) {
> >                 myNovoLogger.finest("Streaming provided object.");
> >                 oos.writeObject(obj);
> >             }
> >             // create an input stream out of the stream
> >             // we put objects into
> > /*            InputStream tis = new ByteArrayInputStream(baos.
> > toByteArray());
> >             /// take the created inputstream and make this the method's
> > bidy
> >             InputStreamEntity inputStreamEntity = new
> > InputStreamEntity(tis);
> >             HttpEntity httpEntity = new InputStreamEntity(tis,
> > tis.available());
> >             //ISSUE in uploading is from here
> >             postMethod.setEntity(inputStreamEntity);*/
> >
> >             InputStreamEntity temp = new InputStreamEntity(new
> > ByteArrayInputStream(baos.toByteArray()));
> >             postMethod.setEntity(temp);
> >
> >
> >
> >             //postMethod.setRequestBody(tis);
> >
> >              RequestConfig config = RequestConfig.custom()
> >                     .setConnectTimeout(20 * 1000)
> >                     .setConnectionRequestTimeout(10* 60 * 1000)
> >                     .setStaleConnectionCheckEnabled(true)
> >                     .setSocketTimeout(10 * 60 * 1000).build();
> >
> >             myNovoLogger.finest("Invoking post method: " +
> > postMethod.getURI().toString());
> >             // execute method on server
> >
> >             PoolingHttpClientConnectionManager connManager = new
> > PoolingHttpClientConnectionManager();
> >             connManager.setValidateAfterInactivity(200);
> >             ConnectionConfig connectionConfig = ConnectionConfig.custom()
> >           .setBufferSize(1000)
> >           .build();
> >            // HttpClientBuilder clientBuilder = HttpClients.custom();
> >           //  clientBuilder.setRetryHandler(new
> > DefaultHttpRequestRetryHandler(3, false));
> >             // HttpClient client = clientBuilder.
> > setDefaultConnectionConfig(connectionConfig).build();
> >             //HttpHost targetHost = new HttpHost(url, 443, "https");
> >             //CloseableHttpResponse response =
> client1.execute(targetHost,
> > postMethod);
> >
> >
> >
> >             this.response =  client.execute(postMethod);  //====> line
> 329
> >             String responseLine = null;
> >
> >             /* HttpHost targetHost = new HttpHost(this.url);
> >              this.response = client.execute(targetHost, postMethod);*/
> >             int rc =0;
> >             try {
> >             HttpEntity entity = response.getEntity();
> >             if (entity != null) {
> >                 entity = new BufferedHttpEntity(entity);
> >             }
> >             if (entity != null) {
> >                 ObjectInputStream instream = new
> ObjectInputStream(entity.
> > getContent());
> >                 try {
> >                         myNovoLogger.finest("Getting response code.");
> >                         myNovoLogger.finest("Got response:" +
> > response.getStatusLine());
> >                         rc = response.getStatusLine().getStatusCode();
> >                         if (rc > 200) {
> >                             myNovoLogger.warning("Response of " + rc);
> >                             myNovoLogger.warning("ResponseBody:" +
> > response.getStatusLine().getReasonPhrase());
> >                         }
> >                         myNovoLogger.finest("Getting input stream.");
> >                         //ois = new ObjectInputStream(entity.
> > getContent());
> >                         //ois = new ObjectInputStream(postMethod.
> > getResponseBodyAsStream());
> >                         myNovoLogger.finest("Done sending request.");
> >                         if(instream != null) {
> >                             myNovoLogger.finest("responseLine instream"+
> > EntityUtils.toString(entity));
> >                             responseLine = (String)
> instream.readObject();
> >                             setResponseLine(responseLine);
> >                             if (Protocol.ACK.equals(responseLine)) {
> >                             Object o = instream.readObject();
> >                             setPayload((Payload) o);
> >                             }
> >                             Object o2 = instream.readObject();
> >                             setDiscardList((HashMap) o2);
> >                         //EntityUtils.consume(response.getEntity());
> >                         }
> >                 } finally {
> >                     myNovoLogger.finest("closing instream");
> >                     if(instream != null) {
> >                         myNovoLogger.finest("closing instream");
> >                     instream.close();
> >                     myNovoLogger.finest("after closing instream");
> >
> >                     }
> >                         this.client.close();
> >                     isValid = false;
> >                     connManager.close();
> >                 }
> >             }
> >         } finally {
> >             myNovoLogger.finest("Attempt to send request "+rc);
> >
> >             return responseLine;
> >         }
> >
> > (Additional info : It works the first call then it fails
> > consecutively...we are upgrading from client 3.2 to 4.5 , have included
> > both core and client in the tomcat load path)
> >
> > Can some one find any problem , any inputs are welcome. I am new to
> > httpclient coding.
> >
> > Thanks
> > Hassan
> >
> > On Sat, Apr 22, 2017 at 9:35 PM, Hassan Khan <hassankhan986@gmail.com>
> > wrote:
> >
> >> Hi Oleg,
> >>
> >> Thanks for answering ... very excited (and humbled)  to hear from you...
> >>
> >> Thanks
> >> Hassan
> >>
> >> On Sat, Apr 22, 2017 at 12:26 PM, Oleg Kalnichevski <olegk@apache.org>
> >> wrote:
> >>
> >>> On Fri, 2017-04-21 at 10:21 -0400, Hassan Khan wrote:
> >>> > Hi all,
> >>> >
> >>> >   We have pinpointed the issue and looks like the code that deals
> >>> > with
> >>> > streaming a object from the client to the server.
> >>> >   With the old code (Httpclient 3.1) we would stream the object
> >>> > through a
> >>> > ObjectOutputStream and on the server side read it through a
> >>> > ObjectInputStream.
> >>> >   But looks like that workflow is not working in the new httpclient
> >>> > 4.5.
> >>> >   Is there any example or resource that specifically shows how
> >>> > streaming
> >>> > objects work ?
> >>> >
> >>>
> >>> See this section of the tutorial
> >>>
> >>> http://hc.apache.org/httpcomponents-client-4.5.x/
> tutorial/html/fundamen
> >>> tals.html#d5e95
> >>> <http://hc.apache.org/httpcomponents-client-4.5.x/
> tutorial/html/fundamentals.html#d5e95>
> >>>
> >>>
> >>> What you most likely want is SerializableEntity
> >>>
> >>> http://hc.apache.org/httpcomponents-core-4.4.x/
> httpcore/apidocs/org/apa
> >>> che/http/entity/SerializableEntity.html
> >>> <http://hc.apache.org/httpcomponents-core-4.4.x/
> httpcore/apidocs/org/apache/http/entity/SerializableEntity.html>
> >>>
> >>> Oleg
> >>>
> >>>
> >>> > Thanks
> >>> > Hassan
> >>> >
> >>> > On Tue, Apr 18, 2017 at 9:27 AM, Hassan Khan <
> hassankhan986@gmail.com
> >>> > >
> >>> > wrote:
> >>> >
> >>> > > Hi ,
> >>> > >
> >>> > > Sorry the log images was filtered out. you can look at the uploaded
> >>> > > image.
> >>> > >
> >>> > > http://imgur.com/a/Nxpcw
> >>> > >
> >>> > > in text the logs say: I/O read timed out.
> >>> > >
> >>> > > Thanks
> >>> > > Hassan
> >>> > >
> >>> > > On Mon, Apr 17, 2017 at 1:35 PM, Gary Gregory <garydgregory@gmail.c
> >>> > > om>
> >>> > > wrote:
> >>> > >
> >>> > > > Hassan,
> >>> > > >
> >>> > > > Your attachments were filtered out. You might want to try
an
> >>> > > > image sharing
> >>> > > > site.
> >>> > > >
> >>> > > > Gary
> >>> > > >
> >>> > > > On Mon, Apr 17, 2017 at 6:16 AM, Hassan Khan <hassankhan986@gmail
> >>> > > > .com>
> >>> > > > wrote:
> >>> > > >
> >>> > > > > +
> >>> > > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > > > Hi All,
> >>> > > > > >
> >>> > > > > >
> >>> > > > > >
> >>> > > > > >    We are upgrading the httpclient in our software
from 3.1
> >>> > > > > > to 4.5 (we
> >>> > > > > > are adding both core and client). But we are having
some
> >>> > > > > > issues in the
> >>> > > > > > client and server communications. We are using
the below
> >>> > > > > > client code
> >>> > > > > > (simplified the code ) to make a  call every 60
secs and we
> >>> > > > > > are getting
> >>> > > > > > localhost not responding after few tries.
> >>> > > > > >
> >>> > > > > > Client code simplified is :
> >>> > > > > >
> >>> > > > > > RequestConfig config = RequestConfig.*custom*()
> >>> > > > > >
> >>> > > > > >               .setConnectTimeout(20 * 1000)
> >>> > > > > >
> >>> > > > > >               .setConnectionRequestTimeout(10*
60 * 1000)
> >>> > > > > >
> >>> > > > > >               .*setStaleConnectionCheckEnabled**(**true**)*
> >>> > > > > >
> >>> > > > > >               .setSocketTimeout(10 * 60 * 1000).build();
> >>> > > > > >
> >>> > > > > > *this*.client =
> >>> > > > > > HttpClients.*custom*().setDefaultRequestConfig(config
> >>> > > > > > ).build();
> >>> > > > > >
> >>> > > > > > HttpResponse response = client.execute(postMethod);
> >>> > > > > >
> >>> > > > > > *int* rc = response.getStatusLine().getStatusCode();
> >>> > > > > >
> >>> > > > > > *if* (rc > 200) {
> >>> > > > > >
> >>> > > > > >       log error
> >>> > > > > >
> >>> > > > > > }
> >>> > > > > >
> >>> > > > > > //using the object stream to read data..
> >>> > > > > >
> >>> > > > > > ois = *new*
> >>> > > > > > ObjectInputStream(response.getEntity().getContent());
> >>> > > > > >
> >>> > > > > > //at the end we close it
> >>> > > > > >
> >>> > > > > > postMethod.releaseConnection();
> >>> > > > > >
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > But still we are getting the following issue:
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > [image: cid:image003.png@01D2B446.A78B7290]
> >>> > > > > >
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > Any recommendations would be appreciated. we have
2 thread
> >>> > > > > > making a
> >>> > > > > > request every 60 secs. When we used a spooling
manager always
> >>> > > > > > one
> >>> > > >
> >>> > > > route was
> >>> > > > > > used..
> >>> > > > > >
> >>> > > > >
> >>> > > > >     May be both thread that are calling the same URL
> >>> > > > > simultaneously are
> >>> > > > > sharing one connection , but they should share different
> >>> > > > > connections..
> >>> > > > >
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > Thanks
> >>> > > > > >
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > *Hassan Khan*
> >>> > > > > >
> >>> > > > > > Software Developer
> >>> > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > >
> >>> > > >
> >>> > > > --
> >>> > > > E-Mail: garydgregory@gmail.com | ggregory@apache.org
> >>> > > > Java Persistence with Hibernate, Second Edition
> >>> > > > <https://www.amazon.com/gp/product/1617290459/ref=as_li_tl?
> >>> > > > ie=UTF8&camp=1789&creative=9325&creativeASIN=1617290459&link
> >>> > > > Code=as2&tag=garygregory-
> >>> > > > 20&linkId=cadb800f39946ec62ea2b1af9fe6a2b8>
> >>> > > >
> >>> > > > <http:////ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=
> >>> > > > am2&o=1&a=1617290459>
> >>> > > > JUnit in Action, Second Edition
> >>> > > > <https://www.amazon.com/gp/product/1935182021/ref=as_li_tl?
> >>> > > > ie=UTF8&camp=1789&creative=9325&creativeASIN=1935182021&link
> >>> > > > Code=as2&tag=garygregory-
> >>> > > > 20&linkId=31ecd1f6b6d1eaf8886ac902a24de418%22>
> >>> > > >
> >>> > > > <http:////ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=
> >>> > > > am2&o=1&a=1935182021>
> >>> > > > Spring Batch in Action
> >>> > > > <https://www.amazon.com/gp/product/1935182951/ref=as_li_tl?
> >>> > > > ie=UTF8&camp=1789&creative=9325&creativeASIN=1935182951&link
> >>> > > > Code=%7B%7BlinkCode%7D%7D&tag=garygregory-20&linkId=%7B%7Bli
> >>> > > > nk_id%7D%7D%22%3ESpring+Batch+in+Action>
> >>> > > > <http:////ir-na.amazon-adsystem.com/e/ir?t=garygregory-20&l=
> >>> > > > am2&o=1&a=1935182951>
> >>> > > > Blog: http://garygregory.wordpress.com
> >>> > > > Home: http://garygregory.com/
> >>> > > > Tweet! http://twitter.com/GaryGregory
> >>> > > >
> >>> > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > > Hassan Khan
> >>> > >
> >>> >
> >>> >
> >>> >
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> >>> For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >>>
> >>>
> >>
> >>
> >> --
> >> Hassan Khan
> >>
> >
> >
> >
> > --
> > Hassan Khan
> >
>
>
>
> --
> Hassan Khan
>



-- 
Hassan Khan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message