lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Optimization and Corruption Issues
Date Thu, 01 Oct 2009 15:47:41 GMT
Would it work to copy your entire index to a new directory, perhaps on a
different machine and optimize *there*? Then copy back to your app. Of
course updates would be lost...
But taking a week to optimize a 20G index seems just plain wrong. Have you
tried playing with the various options to see if you can get better
performance? And/or allocating more memory to the JVM?

Of course I'm not very familiar with 2.0 performance, so.....

Best
Erick

On Thu, Oct 1, 2009 at 11:40 AM, lowfreq <hughmorrison@hotmail.com> wrote:

>
> I have a Lucene index that is very large in size.
> It was created using a pre 2.1 version of Lucene.net 2.0.0.4.
>
> The index is currently almost 20 GB, and has almost 7000 segment files.
> The problem I am having is that I need to optimize it, and cant do this
> without the search functionality of my app being down for a week.
>
> I used the Luke tool from getopt.org and it worked flawlessly, optimizing
> the index in just over 2 hours. Problem is that my search cannot use it,
> and
> the error states Unknown Format Version errors, or just plain nothing
> found.
>
> I understand that versions of Lucene that are newer than what the index was
> built and is searched with can cause problems.
>
> What can I do to make this work? I have tried older versions of Luke, 0.7
> was the oldest I could lay hands on, but even it uses a newer version of
> Lucene.
>
> My index version shows as 633103800023469045. The version the index is
> written as after optimizing with Luke 7.0 is 633103800023469057.
>
> Any help here would be awesome!
>
> Thank you,
>
> Hugh
>
> --
> View this message in context:
> http://www.nabble.com/Optimization-and-Corruption-Issues-tp25697034p25697034.html
> Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

Mime
View raw message