lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Victor Hadianto <vict...@nuix.com.au>
Subject Re: parallizing index building
Date Mon, 30 Jun 2003 23:59:17 GMT
> Where can I find any sample code or documentation about merging a set of
> small indexes into one big index?

It's very simple, just add the index directorioes to your IndexWriter using 
the addIndexes() method, call optimize() and then close().

That's all.


> Lixin

victor

>
> -----Original Message-----
> From: Doug Cutting [mailto:cutting@lucene.com]
> Sent: Monday, June 30, 2003 10:24 AM
> To: Lucene Users List
> Subject: Re: parallizing index building
>
> Marc Dumontier wrote:
> > I'm indexing 500 XML files each ~150Mb on an 8 CPU machine.
> >
> > I'm wondering what the best strategy for making maximum use of resources
>
> is. I have the tweaked the single process indexer to index 5000 records
> (not files) in memory before writing out to disk.
>
> > Should i create an IndexThread and share the IndexWriter object across 5
>
> threads..then monitor when one ends to start another, etc. Or should i
> create difference indexes then to a series of merges.
>
> Creating multiple indexes in parallel and then merging them at the end
> will probably be fastest.
>
> Doug
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org

-- 
Victor Hadianto

NUIX Pty Ltd
Level 8, 143 York Street, Sydney 2000
Phone: (02) 9283 9010
Fax:   (02) 9283 9020

This message is intended only for the named recipient. If you are not the
intended recipient you are notified that disclosing, copying, distributing
or taking any action in reliance on the contents of this message or
attachment is strictly prohibited.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message