Return-Path: Delivered-To: apmail-lucene-solr-user-archive@minotaur.apache.org Received: (qmail 94331 invoked from network); 20 May 2010 19:17:48 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 20 May 2010 19:17:48 -0000 Received: (qmail 50400 invoked by uid 500); 20 May 2010 19:17:46 -0000 Delivered-To: apmail-lucene-solr-user-archive@lucene.apache.org Received: (qmail 50310 invoked by uid 500); 20 May 2010 19:17:46 -0000 Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-user@lucene.apache.org Delivered-To: mailing list solr-user@lucene.apache.org Received: (qmail 50302 invoked by uid 99); 20 May 2010 19:17:46 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 20 May 2010 19:17:46 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=10.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [199.85.64.7] (HELO mlwlb.globeandmail.ca) (199.85.64.7) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 20 May 2010 19:17:38 +0000 X-ASG-Debug-ID: 1274383037-29db03240000-UCWVJZ X-Barracuda-URL: http://199.85.64.7:8000/cgi-bin/mark.cgi Received: from GMEXHUB1.globeandmail.net (unknown [10.100.43.209]) by mlwlb.globeandmail.ca (Spam & Virus Firewall) with ESMTP id 3624B78CE27 for ; Thu, 20 May 2010 15:17:17 -0400 (EDT) Received: from GMEXHUB1.globeandmail.net ([10.100.43.209]) by mlwlb.globeandmail.ca with ESMTP id 40pLhl03V3XGtMoo for ; Thu, 20 May 2010 15:17:17 -0400 (EDT) Received: from GMEXMBS2.globeandmail.net ([fe80::8c9:5dc:9486:5691]) by GMEXHUB1.globeandmail.net ([::1]) with mapi; Thu, 20 May 2010 15:17:16 -0400 From: "Nagelberg, Kallin" To: "'solr-user@lucene.apache.org'" Date: Thu, 20 May 2010 15:17:16 -0400 X-ASG-Orig-Subj: RE: Machine utilization while indexing Subject: RE: Machine utilization while indexing Thread-Topic: Machine utilization while indexing Thread-Index: Acr4ULXSvCgpk/51QHCwhStc1K/P1AAACDbw Message-ID: <13D828CF2C5A6D4597111B6E6571FD63090063C767@GMEXMBS2.globeandmail.net> References: <4BF54EE8.7000308@gmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: UNKNOWN[10.100.43.209] X-Barracuda-Start-Time: 1274383037 X-Barracuda-Virus-Scanned: by Barracuda Spam & Virus Firewall at globeandmail.ca X-Barracuda-Spam-Score: -1002.00 X-Barracuda-Spam-Status: No, SCORE=-1002.00 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=1000.0 X-Virus-Checked: Checked by ClamAV on apache.org StreamingUpdateSolrServer already has multiple threads and uses multiple co= nnections under the covers. At least the api says ' Uses an internal MultiT= hreadedHttpConnectionManager to manage http connections'. The constructor a= llows you to specify the number of threads used, http://lucene.apache.org/s= olr/api/org/apache/solr/client/solrj/impl/StreamingUpdateSolrServer.html#St= reamingUpdateSolrServer(java.lang.String, int, int) .=20 -Kallin Nagelberg -----Original Message----- From: Chris Hostetter [mailto:hossman_lucene@fucit.org]=20 Sent: Thursday, May 20, 2010 3:14 PM To: solr-user@lucene.apache.org Subject: Re: Machine utilization while indexing I'm really only guessing here, but based on your description of what you=20 are doing it sounds like you only have one thread streaming documents to=20 solr (via a single StreamingUpdateSolrServer instance which creates a=20 single HTTP connection) Have you at all attempted to have parallel threads in your client initiate= =20 parallel connections to Solr via multiple instances of=20 StreamingUpdateSolrServer objects?) -Hoss