lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "peter velthuis" <peter...@gmail.com>
Subject Indexing very slow.
Date Mon, 03 Jul 2006 08:52:15 GMT
When i start the program its fast.. about 10 docs per second. but
after about 15000 it slows down very much. Now it does 1 doc per
second and it is at nr# 40 000 after a whole night indexing. These are
VERY small docs with very little information.. THis is what and how i
index it:

      Document doc = new Document();
                                 doc.add(new Field("field1", field1,
Field.Store.YES,
                        Field.Index.TOKENIZED));
                                 doc.add(new Field("field2", field2,
Field.Store.YES,
                        Field.Index.TOKENIZED));
                                 doc.add(new Field("field3", field3,
Field.Store.YES,
                        Field.Index.TOKENIZED));
                                 doc.add(new Field("field4", field4,
Field.Store.YES,
                        Field.Index.TOKENIZED));
                                 doc.add(new Field("field5", field5,
Field.Store.YES,
                        Field.Index.TOKENIZED));
                                doc.add(new Field("field6", field6,
Field.Store.YES,
                        Field.Index.TOKENIZED));
                                doc.add(new Field("contents",
contents, Field.Store.NO,
                        Field.Index.TOKENIZED));



and this:


    String indexDirectory = "lucdex2";

    private void indexDocument(Document document) throws Exception {
        Analyzer analyzer  = new StandardAnalyzer();
        IndexWriter writer = new IndexWriter(indexDirectory, analyzer, false);
      //  writer.setUseCompoundFile(true);
        writer.addDocument(document);
        writer.optimize();
        writer.close();



I read the data out mysql database.. but that cant be the problem..
since data is in memory.

Also i use cygwin, when i try indexing on windows in a program like
netbeans or BlueJ it crashes windows after about 5000 docs. it sais
"beep" and a complete shutdown...

Peter

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message