lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andy Hind" <andy.h...@alfresco.org>
Subject RE: weird behavior of IndexReader.indexExists()
Date Tue, 09 May 2006 11:46:44 GMT
Hi

I think I have discovered this too.
It is on my list of issues to raise .... 

The index exist test looks for the segment file.
When the index is committing, and you are unlucky, this file may not be
found as the new segments file replaces the old one. The result is the
index appears not to exist.

Regards

Andy

-----Original Message-----
From: wenjie zheng [mailto:wenjie.zheng@gmail.com] 
Sent: 08 May 2006 18:57
To: java-dev@lucene.apache.org
Subject: Re: weird behavior of IndexReader.indexExists()

This happens sometimes when number of docs is over 2000. So it's kinda
of
random.

Wenjie

On 5/8/06, wenjie zheng <wenjie.zheng@gmail.com> wrote:
>
> I created an index with more than 30,000 text files.
> I used indexExists() to determine either to create a new index or to
add
> docs to the existing index.
>
> But when the num of docs in the index was over 3,000 (sometimes 3,400,
> sometimes 3,200), the indexExists function returns false, so I ended
up
> recreating a new index.
>
> Here is my code:
> if index exists, we will add files to it, otherwise, create a new
index.
> In either case, an IndexingThread will be spawn to do that.
>          if(IndexReader.indexExists(indexDir)){
>                 logger.info("Working on existing index ...");
>                 IndexingThread.startIndexingThread(Username, new
> File(propsFile), new File(indexDir), docs,
>                               new StandardAnalyzer(), false);
>           }else{
>                 logger.info("Create a new index ...");
>                 IndexingThread.startIndexingThread(Username, new
> File(propsFile), new File(indexDir), docs,
>                                new StandardAnalyzer(), true);
>            }
>
>
> inside the startIndexingThread function, I am calling the following
> function to add files to the index:
>     /**
>      * Add an array of Files to an index
>      *
>      * @param propsFile the properties file
>      * @param indexDir  the folder where index files will be created
in
>      * @param docs      an array of Files to be add to the index
>      * @param analyzer  any Analyzer object
>      */
>     public void addFiles(File propsFile, File indexDir, File[] docs,
> Analyzer analyzer, boolean overwrite) throws Exception {
>         Properties props = new Properties(new
FileInputStream(propsFile));
>
>         if(overwrite || IndexReader.indexExists(indexDir)){ //either
> overwrite or working on an existing index
>             Directory index = FSDirectory.getDirectory(indexDir,
> overwrite);
>             IndexWriter writer = new IndexWriter(index, analyzer,
> overwrite);
>
>             FileIndexer indexer = new FileIndexer(props);
>
>             long start = new Date().getTime();
>             indexer.index(writer, docs);
>             writer.optimize();
>             writer.close ();     // close the writer
>             index.close();      // close the index Directory
>             long end = new Date().getTime();
>
>             logger.info("Total time: " + (end - start) + " ms");
>
>         }else{
>             logger.error("Index files are not found: " +
> indexDir.getAbsolutePath() + ", overwrite = false");
>         }
>     }
>
> Thanks,
> Wenjie
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message