lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wenjie zheng" <wenjie.zh...@gmail.com>
Subject Re: weird behavior of IndexReader.indexExists()
Date Mon, 08 May 2006 17:57:11 GMT
This happens sometimes when number of docs is over 2000. So it's kinda of
random.

Wenjie

On 5/8/06, wenjie zheng <wenjie.zheng@gmail.com> wrote:
>
> I created an index with more than 30,000 text files.
> I used indexExists() to determine either to create a new index or to add
> docs to the existing index.
>
> But when the num of docs in the index was over 3,000 (sometimes 3,400,
> sometimes 3,200), the indexExists function returns false, so I ended up
> recreating a new index.
>
> Here is my code:
> if index exists, we will add files to it, otherwise, create a new index.
> In either case, an IndexingThread will be spawn to do that.
>          if(IndexReader.indexExists(indexDir)){
>                 logger.info("Working on existing index ...");
>                 IndexingThread.startIndexingThread(Username, new
> File(propsFile), new File(indexDir), docs,
>                               new StandardAnalyzer(), false);
>           }else{
>                 logger.info("Create a new index ...");
>                 IndexingThread.startIndexingThread(Username, new
> File(propsFile), new File(indexDir), docs,
>                                new StandardAnalyzer(), true);
>            }
>
>
> inside the startIndexingThread function, I am calling the following
> function to add files to the index:
>     /**
>      * Add an array of Files to an index
>      *
>      * @param propsFile the properties file
>      * @param indexDir  the folder where index files will be created in
>      * @param docs      an array of Files to be add to the index
>      * @param analyzer  any Analyzer object
>      */
>     public void addFiles(File propsFile, File indexDir, File[] docs,
> Analyzer analyzer, boolean overwrite) throws Exception {
>         Properties props = new Properties(new FileInputStream(propsFile));
>
>         if(overwrite || IndexReader.indexExists(indexDir)){ //either
> overwrite or working on an existing index
>             Directory index = FSDirectory.getDirectory(indexDir,
> overwrite);
>             IndexWriter writer = new IndexWriter(index, analyzer,
> overwrite);
>
>             FileIndexer indexer = new FileIndexer(props);
>
>             long start = new Date().getTime();
>             indexer.index(writer, docs);
>             writer.optimize();
>             writer.close ();     // close the writer
>             index.close();      // close the index Directory
>             long end = new Date().getTime();
>
>             logger.info("Total time: " + (end - start) + " ms");
>
>         }else{
>             logger.error("Index files are not found: " +
> indexDir.getAbsolutePath() + ", overwrite = false");
>         }
>     }
>
> Thanks,
> Wenjie
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message