hc-httpclient-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William Speirs <wspe...@apache.org>
Subject Re: Help with Multiple Connections
Date Sun, 19 Feb 2012 15:45:33 GMT
Can you post the stack trace.

Bill-
On Feb 18, 2012 8:01 PM, "m0j0j0j0" <silvio.technologies@gmail.com> wrote:

>
> Hello all, ive been developing an automation tool for myself.
>
> After modifying the example code for a multithreaded get request, and
> setting up all my logic,
> i had everything working great. Testing with 1340 urls everything worked
> fine.
>
> Further testing i loaded over 3000 urls and connection pool closed
> IllegalStateException
> Ive been playing with this for a day or 2.
>
> Keep getting an exception that i cant seem to track down!
>
> here is my httpget request runnable with all this testing ive done i
> completely removed my logic
> and placed a simple alive check for testing purposes..
>
>        public void run() {
>                HttpEntity entity = null;
>                ResponseHandler<byte[]> handler = new
> ResponseHandler<byte[]>() {
>                    public byte[] handleResponse(
>                            HttpResponse response) throws
> ClientProtocolException,
> IOException {
>                        HttpEntity entity = response.getEntity();
>                        if (entity != null) {
>
>  mdl1.setValueAt(response.getStatusLine().getStatusCode(), id-1,
> 0);
>                            return EntityUtils.toByteArray(entity);
>                        } else {
>                                mdl1.setValueAt("TO", id-1, 0);
>                                httpget.abort();
>                            return null;
>                        }
>                    }
>                };
>
>                try {
>                        byte[] response = httpClient.execute(httpget,
> handler);
>                        EntityUtils.consume(entity);
>                } catch (ClientProtocolException e) {
>                        mdl1.setValueAt("TO", id-1, 0);
>                        System.out.println("CLIENTPROTOEXE " +
> e.getMessage() + "\n" +
> e.getCause());
>                        httpget.abort();
>                        //e.printStackTrace();
>                } catch (IOException e) {
>                        mdl1.setValueAt("TO", id-1, 0);
>                        System.out.println(e.getMessage());
>                        httpget.abort();
>                        //e.printStackTrace();
>                }
>                try {
>                        EntityUtils.consume(entity);
>                } catch (IOException e) {
>
>                        // TODO Auto-generated catch block
>                        System.out.println("CONSUME + " + e.getMessage());
>                        //e.printStackTrace();
>                }
> }
>
>
>
> in my starting thread ive tried multiple things.  My last attempt was to
> try
> an executor
> to limit the ammount of threads, this seem to work until almost halfway
> through the list
> and i keep getting java.lang.IllegalStateException: Connection pool shut
> down
>
>                                for (int j = 0; j < dmdl1.getRowCount();
> j++) {
>                                        exec.execute(new
> GetThread(httpclient, new HttpGet(dmdl1.getValueAt(j,
> 2).toString()), j + 1, dmdl1, (ThreadSafeClientConnManager) cm));
>                                }
>
>
> My original try was to use the traditional method which i though would do
> the work for me.
> after creating each getrequest thread i used the code below.  Which was
> working fine
> when i was doing testing and loaded 1340 urls, but when i try over 3000
> urls
> it fails.  Im kindof
> new to java mostly im out of practice from not using it for quite some time
> but im eager to get
> back into it.
>
> //            for (int j = 0; j < threads.length; j++) {
> //                      //System.out.println(cm.getConnectionsInPool() +
> "-- thread
> # " + j + " " + dmdl1.getValueAt(j, 2).toString());
> //                      threads[j].start();
> //            }
> //
> //            // join the threads
> //            for (int j = 0; j < threads.length; j++) {
> //                  threads[j].join();
> //                     System.out.println("Joined id " + j);
> //            }
>
>
> Im using HttpClient 4.1.2 in my project, please any insight on how to
> initiate these connections
> or manage them would be helpful.
>
>                        SchemeRegistry schemeRegistry = new
> SchemeRegistry();
>                                schemeRegistry.register(
>                                        new Scheme("http", 80,
> PlainSocketFactory.getSocketFactory()));
>
>                                ClientConnectionManager cm = new
> ThreadSafeClientConnManager(schemeRegistry);
>                                HttpClient httpClient = new
> DefaultHttpClient(cm);
>
>            ((ThreadSafeClientConnManager) cm).setMaxTotal(50);
>        ((ThreadSafeClientConnManager) cm).setDefaultMaxPerRoute(10);
>        HttpClient httpclient = new DefaultHttpClient(cm);
>        httpclient.getParams().setParameter("http.socket.timeout", new
> Integer(10000));
>
> I thought using threadsafeconnmanager it would do all the work for me:(
>
> Then i thought of a way to have a locking thread, that waits for
> connectionmanager.getconnectionsinpool to drop below half th total and
> start
> initiating more connections.  Again im kinda confused i'd love any help you
> can provide.
>
> Thank you for your time.
>
>
>
>
>
>
> --
> View this message in context:
> http://old.nabble.com/Help-with-Multiple-Connections-tp33350254p33350254.html
> Sent from the HttpClient-User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message