Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: local policy)
Message-ID: <43E0A676.5000405@i-hypergrid.com>
Date: Wed, 01 Feb 2006 14:15:50 +0200
From: Volodymyr Bychkoviak <vbychkoviak@i-hypergrid.com>
User-Agent: Thunderbird 1.5 (Windows/20051201)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Re: Memory problem
References: <e97a509fb54c459aa9f81191c764d158@danicorp.com>
 <005201c62719$5e199b80$6500a8c0@leondell>
In-Reply-To: <005201c62719$5e199b80$6500a8c0@leondell>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

As long as you have many document in index there can many unique terms 
in index.
Every 128th term(by default) is written to term info index for faster 
term lookup.
This info is loaded entirely to memory when searching so this can 
increase memory usage.
Note that this does not depends on number of documents in index, it 
depends on number of unique terms in index.

This can be changed by setting higher value in 
indexWriter.setTermIndexInterval();
Be aware of setting this value too hight because search performance will 
degrade.
NOTE: this options is available only in Lucene 1.9.

Also it can depend on number of fields in document, the way you process 
them (store, index, tokenize etc.)


Leon Chaddock wrote:
> Hi All,
>
> We have a lucene index of over 10 000 000 docs at this time.
> When we try and run a search we get
> java.lang.OutOfMemoryError: Java heap space
>
> We have tried setting the xmx settings to 1gb but to no avail (the box 
> has 4gb of memory available) . IS there any guidance on handling 
> memory or has anyone had similar problems before that could help?
>
> Many thanks
>
> Leon
>
> ----- Original Message ----- From: "Pradeep Sharma" 
> <pradeep@danicorp.com>
> To: <java-user@lucene.apache.org>
> Sent: Wednesday, February 01, 2006 2:03 AM
> Subject: Greetings and my first question - Is it a good practise to 
> store application configuration in Lucene
>
>
>
>
> I have just joined this user group, but I probably will be asking 
> questions / contributing for a while now as I am starting to work on a 
> product which will use Lucene exclusively.
>
> Still in the designing phase, and I see that we need to manage several 
> user / application specific configurations and I am exploring the idea 
> of storing the configuration information also in the Index, may be 
> create a separate index just for the configuration, because each 
> module of the application will have access to Lucene classes.
>
> I know technically this can be done, but are there any best practises 
> which discourage this?
>
> Thanks in advance.
> -Pradeep
>
>
>
> -------------------------------------------------------------------------------- 
>
>
>
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.1.375 / Virus Database: 267.14.25/246 - Release Date: 
> 30/01/2006
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

-- 
regards,
Volodymyr Bychkoviak


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org