Return-Path: Delivered-To: apmail-jakarta-lucene-user-archive@www.apache.org Received: (qmail 74569 invoked from network); 23 Jul 2004 12:34:25 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur-2.apache.org with SMTP; 23 Jul 2004 12:34:25 -0000 Received: (qmail 33500 invoked by uid 500); 23 Jul 2004 12:34:19 -0000 Delivered-To: apmail-jakarta-lucene-user-archive@jakarta.apache.org Received: (qmail 33473 invoked by uid 500); 23 Jul 2004 12:34:19 -0000 Mailing-List: contact lucene-user-help@jakarta.apache.org; run by ezmlm Precedence: bulk List-Unsubscribe: List-Subscribe: List-Help: List-Post: List-Id: "Lucene Users List" Reply-To: "Lucene Users List" Delivered-To: mailing list lucene-user@jakarta.apache.org Received: (qmail 33456 invoked by uid 99); 23 Jul 2004 12:34:19 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [203.199.26.74] (HELO daakghar.controlnet.co.in) (203.199.26.74) by apache.org (qpsmtpd/0.27.1) with SMTP; Fri, 23 Jul 2004 05:34:16 -0700 Received: from karthik ([192.168.4.1]) by dakiya.controlnet.co.in (Netscape Messaging Server 4.15) with ESMTP id I1B26300.S3U for ; Fri, 23 Jul 2004 18:16:51 +0530 From: "Karthik N S" To: "Lucene Users List" Subject: RE: Large index files Date: Fri, 23 Jul 2004 18:14:23 +0530 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 In-Reply-To: X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Hi I think (a) would be a better choice [I have done it on Linux upt to 7GB , it's pretty faster then doing the same on win2000 PF] with regards Karthik -----Original Message----- From: Rupinder Singh Mazara [mailto:rsmazara@ebi.ac.uk] Sent: Friday, July 23, 2004 5:55 PM To: Lucene Users List Subject: Large index files Hi all I am using lucene to index a large dataset, it so happens 10% of this data yields indexes of 400MB, in all likelihood it is possible the index may go upto 7GB. My deployment will be on a linux/tomcat system, what will be a better solution a) create one large index and hope linux does not mind b) generate 7-10 indexes based on some criteria and glue them together using MultiReader, in this case I may cross the MAX file handles limit of Tomcat ? regards --------------------------------------------------------------------- To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org For additional commands, e-mail: lucene-user-help@jakarta.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org For additional commands, e-mail: lucene-user-help@jakarta.apache.org