Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (hermes.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding;
  b=gMzB6xLCDcP52etrk0KYbLE+aOBHLBFb5bKrU2FEF5baLGaus2BGyxJduJwcN0HsEJOkVcflXCai8TlY19YJexmyAzxr0NBW34cvqeXfAfbsIbR3dRx+4YVHb0tjgPCGq08LHUCBahkHlssQH+wBovLLHGkBJSGNQLo+YIyfPhs=
  ;
Message-ID: <20050610005238.82891.qmail@web50601.mail.yahoo.com>
Date: Thu, 9 Jun 2005 17:52:38 -0700 (PDT)
From: Chris Collins <chris_j_collins@yahoo.com>
Subject: Re: Optimizing indexes with mulitiple processors?
To: java-user@lucene.apache.org, Bill Au <bill.w.au@gmail.com>
In-Reply-To: <3b5f720305060914363281d64b@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit

I found with a fast RAID controller that I can easily be CPU bound, some of the
io is related to latency.  You can hide the latency by having overlapping IO
(you get that with multiple indexers going on at the same time).

I think there possibly could be more horsepower you can get out of the inverter
and merge aspects of the indexing.  I am currently jprobeing this at the
moment.

If your using high latency disks (such as a filer) during merge you may want to
consider increasing the size of the buffers to reduce the amount of rpc's to
the filer....however my previous attempts to change this failed.

C 

--- Bill Au <bill.w.au@gmail.com> wrote:

> Optimize is disk I/O bound.  So I am not sure what multiple CPUs will buy
> you.
> 
> Bill
> 
> On 6/9/05, Kevin Burton <burton@rojo.com> wrote:
> > Is it possible to get Lucene to do an index optimize on multiple
> > processors?
> > 
> > Its a single threaded algorithm currently right?
> > 
> > Its a shame since I have a quad  machine but I'm only using 1/4th of the
> > capacity.  Thats a heck of a performance hit.
> > 
> > Kevin
> > 
> > --
> > 
> > 
> > Use Rojo (RSS/Atom aggregator)! - visit http://rojo.com.
> > See irc.freenode.net #rojo if you want to chat.
> > 
> > Rojo is Hiring! - http://www.rojonetworks.com/JobsAtRojo.html
> > 
> >    Kevin A. Burton, Location - San Francisco, CA
> >       AIM/YIM - sfburtonator,  Web - http://peerfear.org/
> > GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412
> > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> > 
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org