Return-Path: Delivered-To: apmail-lucene-java-user-archive@www.apache.org Received: (qmail 1654 invoked from network); 11 Feb 2008 12:46:47 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 11 Feb 2008 12:46:47 -0000 Received: (qmail 16201 invoked by uid 500); 11 Feb 2008 12:46:33 -0000 Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org Received: (qmail 16171 invoked by uid 500); 11 Feb 2008 12:46:33 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 16126 invoked by uid 99); 11 Feb 2008 12:46:33 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Feb 2008 04:46:33 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [208.97.132.83] (HELO spunkymail-a19.g.dreamhost.com) (208.97.132.83) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Feb 2008 12:46:00 +0000 Received: from [192.168.0.3] (adsl-074-229-189-244.sip.rmo.bellsouth.net [74.229.189.244]) by spunkymail-a19.g.dreamhost.com (Postfix) with ESMTP id CB52B11E00 for ; Mon, 11 Feb 2008 04:46:09 -0800 (PST) Message-Id: From: Grant Ingersoll To: java-user@lucene.apache.org In-Reply-To: <629CBB07E28C934D8CA87EF193170E4C01D65532@tshuspaphimbx01.ERF.THOMSON.COM> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v915) Subject: Re: large term vectors Date: Mon, 11 Feb 2008 07:46:07 -0500 References: <629CBB07E28C934D8CA87EF193170E4C01D65532@tshuspaphimbx01.ERF.THOMSON.COM> X-Mailer: Apple Mail (2.915) X-Virus-Checked: Checked by ClamAV on apache.org Hi Marc, Can you give more info about what your field properties are? Your subject line implies you are storing term vectors, is that the case? Also, what version of Lucene are you using? Cheers, Grant On Feb 8, 2008, at 10:51 AM, wrote: > Hi, > > > > I have a large index which is around 275GB. As I search different > parts > of the index, the memory footprint grows with large byte arrays being > stored. They never seem to get unloaded or GC'ed. Is there any way to > control this behavior so that I can periodically unload cached > information? > > > > The nature of the data being indexed doesn't allow me to reduce the > number of terms per field, although I might be able to reduce the > number > of overall fields (I have some which aren't currently being searched > by). > > > > I've just begun investigating and profiling the problem, so I don't > have > a lot of details at this time. Any support would be extremely welcome. > > > > Thanks, > > > > Marc Dumontier > Manager, Software Development > Thomson Scientific (Canada) > 1 Yonge Street, Suite 1801 > Toronto, Ontario M5E 1W7 > > > > Direct +1 416 214 3448 > Mobile +1 416 454 3147 > > > -------------------------- Grant Ingersoll http://lucene.grantingersoll.com http://www.lucenebootcamp.com Lucene Helpful Hints: http://wiki.apache.org/lucene-java/BasicsOfPerformance http://wiki.apache.org/lucene-java/LuceneFAQ --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org For additional commands, e-mail: java-user-help@lucene.apache.org