Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: domain of markrmiller@gmail.com designates
 64.233.166.183 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=cd1a4brykDMr56Iqtf8qW0FczCN3B2kofpx5WIMfhD9WOs8gOcdEZlvMxVyHpud8RuvlwFJtnhAHdkUDMb7XewcqXSqvVnhtZRgHlerkY1eIi6+ukZWzMn+zQnLEa5eVZPY1qh6XSqTZtB6oV56acU06AwEAL6Q7OwrZwyeyBWk=
Message-ID: <ab0709bf0610131841i3a88028jf3a9aec044504355@mail.gmail.com>
Date: Fri, 13 Oct 2006 21:41:38 -0400
From: "Mark Miller" <markrmiller@gmail.com>
To: java-user@lucene.apache.org
Subject: Re: Large index question
In-Reply-To: 
 <55F9EF953014514EB6BEE1F5198CEEB44897CD@exchange.slc.mainstreamdata.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_54006_25247183.1160790098842"
References: 
 <55F9EF953014514EB6BEE1F5198CEEB44897CD@exchange.slc.mainstreamdata.com>

------=_Part_54006_25247183.1160790098842
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

I recently played around with a 2 million doc index of docs that averaged
between 2-10k. The system had 4 gig of ram and a 3 gig dual core proc (not
using a parallel searcher to take advantage of the extra core)...pretty
beefy, but with 4 times the docs your talking about. I didn't see a query
that took over a second without a sort.

A similar setup on a single core 3200+ AMD 64 with a gig of ram was also
blazingly fast (no sorts involved again).

- Mark

On 10/12/06, Scott Smith <ssmith@mainstreamdata.com> wrote:
>
> Supposed I want to index 500,000 documents (average document size is
> 4kBs).  Let's assume I create a single index and that the index is
> static (I'm not going to add any new documents to it).  I would guess
> the index would be around 2GB.
>
>
>
> Now, I do searches against this on a somewhat beefy machine (2GB RAM,
> Core 2 Duo, Windows XP).  Does anyone have any idea what kinds of search
> times I can expect for moderately complicated searches (several sets of
> keywords against several fields)?  Are there things I can do to increase
> search performance?  For example, does Lucene like lots of RAM, lots of
> CPU, faster HD, all of the above?  Am I better splitting the index file
> into 2 (N?) versions and search on multiple indexes simultaneously?
>
>
>
> Anyone have any thoughts about this?
>
>
>
> Scott
>
>
>
>
>

------=_Part_54006_25247183.1160790098842--