Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: neutral (asf.osuosl.org: local policy)
Message-ID: <47952.192.223.163.5.1122902104.squirrel@192.223.163.5>
In-Reply-To: <42ED9504.8040101@cosmicrealms.com>
References: <42ED9504.8040101@cosmicrealms.com>
Date: Mon, 1 Aug 2005 09:15:04 -0400 (EDT)
Subject: Re: Any problems with a failed IndexWriter optimize call?
From: "Tony Schwartz" <tony@simpleobjects.com>
To: java-user@lucene.apache.org
User-Agent: SquirrelMail/1.4.3a
MIME-Version: 1.0
Content-Type: text/plain;charset=iso-8859-1
Content-Transfer-Encoding: 8bit
Importance: Normal

Your index should be fine.  You could use "luke" if you want to remove any dangling
files not in use.  Just run optimize again to fix it all up...  You might want to
allocate more memory that 175 though.  Depending on your document sizes and how
efficiently and fast you want lucene to work to index the data, you will want to give
lucene plenty of memory to work with.

Tony Schwartz
tony@simpleobjects.com


> Hello! I am using Lucene 1.4.3
>
> I'm building a Lucene index, that will have about 25 million documents
> when it is done.
> I'm adding 250,000 at a time.
>
> Currently there is about 1.2Million in there, and I ran into a problem.
> After I had added a batch of 250,000 I go a 'java.lang.outOfMemory'
> threw by writer.optimize(); (a standard IndexWriter)
>
> The exception caused my program to quit out, and it didn't call
> 'writer.close();'
>
> First, with it dying in the middle of an .optimize() is there any chance
> my index is corrupted?
>
> Second, I know I can remove the /tmp/lucene*.lock file to remove the
> lock in order to add more, but is it safe to do that?
>
> I've since figured out that I can pass -Xmx to the 'java' program in
> order to increase the maximum amount of RAM.
> It was using the default of 64M, I plan on increasing that to 175M to
> start with.
> That should solve the memory problems (I can allocate more if necessary
> down the line).
>
> Lastly, when I go back, open it again, and add another 250,000 and then
> call optimize again, will a failed previous optimize hurt the index at all?
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org