Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: domain of markrmiller@gmail.com designates
 66.249.82.224 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:user-agent:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding;
        b=h4VIufC1xwQI1NS20YAE7xcMEHC3cufIajP6oV0BdpygtQzIaXJokQHi7phPIBDShbOwDXo72GaPLc8olgYHokHsExekzr4tJb8cj9c6oIIwZa3j59EJYMCmzwmkc49eH4lf74E98HI6iAP+BGy60zjvt205mNyx8KgeuVLrksw=
Message-ID: <44DD0057.1030606@gmail.com>
Date: Fri, 11 Aug 2006 18:10:31 -0400
From: Mark Miller <markrmiller@gmail.com>
User-Agent: Thunderbird 1.5.0.5 (Windows/20060719)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: 30 milllion+ docs on a single server
References: 
 <OF9D529309.F7DE059A-ON882571C6.00642C64-882571C6.006F2D56@il.ibm.com>
 <Pine.LNX.4.58.0608111136530.10034@hal.rescomp.berkeley.edu>
In-Reply-To: <Pine.LNX.4.58.0608111136530.10034@hal.rescomp.berkeley.edu>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I've made a nice little archive application with lucene. I made it to 
handle our largest need: 2.5 million docs or so on a single server. Now 
the powers that be say: lets use it for a 30+ million document archive 
on a single server! (each doc size maybe 10k max...as small as a 1 or 
2k) Please tell me why we are in trouble...please tell me why we are 
not. I have tested up to 2 million docs without much trouble but 30 
million...the average search will include a sort on a field as 
well...can I search 30+ million docs with a sort? Man am I worried about 
that. Maybe the server will have 8 procs and 12 billion gigs of RAM. 
Mabye. Even still, Tomcat seems to be able to launch with a max of 1.5 
or 1.6 gig of Ram in Windows. What do you think? 30 million+ sounds like 
too much of a load to me for a single server. Not that they care what I 
think...I only wrote the thing (man I hate my job, offer me a new one :) 
)...please...comments?

Cheers,

Miserable Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org