Return-Path: Delivered-To: apmail-lucene-java-user-archive@www.apache.org Received: (qmail 1497 invoked from network); 3 Jul 2006 08:52:45 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 3 Jul 2006 08:52:45 -0000 Received: (qmail 950 invoked by uid 500); 3 Jul 2006 08:52:39 -0000 Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org Received: (qmail 912 invoked by uid 500); 3 Jul 2006 08:52:38 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 900 invoked by uid 99); 3 Jul 2006 08:52:38 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jul 2006 01:52:38 -0700 X-ASF-Spam-Status: No, hits=0.5 required=10.0 tests=DNS_FROM_RFC_ABUSE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: domain of peterv85@gmail.com designates 66.249.92.170 as permitted sender) Received: from [66.249.92.170] (HELO ug-out-1314.google.com) (66.249.92.170) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jul 2006 01:52:37 -0700 Received: by ug-out-1314.google.com with SMTP id u2so1413352uge for ; Mon, 03 Jul 2006 01:52:15 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=U2u9qLf+zgLaczbS4QV5y3m1s0Ejl9JbKI2uSkCDAuGoSKGaZyRkn1FsgKCR50fRYy9mDT9oCKNISfxQd8ck2kCjiPd+XJ74DJQlxEWyxFZcgkXhvvmoPLoBgoQ7uCocZje/Lp3obZH2ODcI7PCeKIVv2D8wwABtC04FEmcyOgc= Received: by 10.67.29.12 with SMTP id g12mr5541578ugj; Mon, 03 Jul 2006 01:52:15 -0700 (PDT) Received: by 10.66.223.5 with HTTP; Mon, 3 Jul 2006 01:52:15 -0700 (PDT) Message-ID: <3d159d980607030152n5d7bd2fbg7b88ea80eaae47d2@mail.gmail.com> Date: Mon, 3 Jul 2006 09:52:15 +0100 From: "peter velthuis" To: java-user@lucene.apache.org Subject: Indexing very slow. MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N When i start the program its fast.. about 10 docs per second. but after about 15000 it slows down very much. Now it does 1 doc per second and it is at nr# 40 000 after a whole night indexing. These are VERY small docs with very little information.. THis is what and how i index it: Document doc = new Document(); doc.add(new Field("field1", field1, Field.Store.YES, Field.Index.TOKENIZED)); doc.add(new Field("field2", field2, Field.Store.YES, Field.Index.TOKENIZED)); doc.add(new Field("field3", field3, Field.Store.YES, Field.Index.TOKENIZED)); doc.add(new Field("field4", field4, Field.Store.YES, Field.Index.TOKENIZED)); doc.add(new Field("field5", field5, Field.Store.YES, Field.Index.TOKENIZED)); doc.add(new Field("field6", field6, Field.Store.YES, Field.Index.TOKENIZED)); doc.add(new Field("contents", contents, Field.Store.NO, Field.Index.TOKENIZED)); and this: String indexDirectory = "lucdex2"; private void indexDocument(Document document) throws Exception { Analyzer analyzer = new StandardAnalyzer(); IndexWriter writer = new IndexWriter(indexDirectory, analyzer, false); // writer.setUseCompoundFile(true); writer.addDocument(document); writer.optimize(); writer.close(); I read the data out mysql database.. but that cant be the problem.. since data is in memory. Also i use cygwin, when i try indexing on windows in a program like netbeans or BlueJ it crashes windows after about 5000 docs. it sais "beep" and a complete shutdown... Peter --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org For additional commands, e-mail: java-user-help@lucene.apache.org