lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Ochoa" <>
Subject Re: Help to solve an issue when upgrading Lucene-Oracle integration to lucene 2.3.1
Date Wed, 07 May 2008 12:07:01 GMT
Hi Michael:
  First thanks a lot for your time.
  See comments below.
>  Is there any way to capture & serialize the actual documents being
>  added (this way I can "replay" those docs to reproduce it)?
  Documents are a column VARCHAR2 from all_source Oracle's System
view, in fact is a table as:
create table test_source_big as (select * from all_source);
>  Are you using threads?  Is autoCommit true or false?
  Oracle JVM uses by default a single Thread model, except that Lucene
is starting a parallel Thread. InfoStream information shows only one
  AutoCommit is false.
  I am creating LuceneWritter with this code:
        IndexWriter writer = null;
        Parameters parameters = dir.getParameters();
        int mergeFactor =
                                                     "" +
        int maxBufferedDocs =
                                                     "" +
        int maxMergeDocs =
                                                     "" +
        int maxBufferedDeleteTerms =
                                                     "" +

        Analyzer analyzer = getAnalyzer(parameters);
        boolean useCompountFileName =
        boolean autoTuneMemory =
        writer =
                new IndexWriter(dir, autoCommitEnable, analyzer, createEnable);
        if (autoTuneMemory) {
            long memLimit =
  ".getIndexWriterForDir - Memory limit for
indexing (Mb): "+memLimit);
        } else
        if (logger.isLoggable(Level.FINE))
   The example pass these relevant parameters:
   So, because AutoTuneMemory is true, instead of setting
MaxBufferedDocs I am setting RAMBufferSizeMB(53) which is calculated
using Oracle SGA free memory.
>  Are you using payloads?
>  Were there any previous exceptions in this IndexWriter before flushing
>  this segment?  Could you post the full infoStream output?
  There is no provious exception. Attached a .trc file generated by
Oracle 11g, it have infoStream information plus logging informartion
from Oracle-Lucene data cartridge.
>  Could you apply the patch below & re-run?  It will likely produce
>  insane amounts of output, but we only need the last section to see
>  which term is hitting the bug.  If that term consistently hits the bug
>  then we can focus on how/when it gets indexed...
  I'll patch my lucene-2.3.1 source and send again the .trc file.
  Also, I am comparing FSDirectory implementation (2.3.1) with my
OJVMDirectory implementation to see changes on how the API of
BufferedIndex[Input|Output].java is used, may be here is the problem.
  For example latest implementation wait an IOException when open an
IndexInput and a file doesn't exists, my code throw a RuntimeException
wich works with Lucene 2.2.x but doesn't work with 2.3.1, this was the
first change to get Lucene-Oracle integration working.
  Best regards. Marcelo.
Marcelo F. Ochoa
Do you Know DBPrism? Look @ DB Prism's Web Site
More info?
Chapter 17 of the book "Programming the Oracle Database using Java &
Web Services"
Chapter 21 of the book "Professional XML Databases" - Wrox Press
Chapter 8 of the book "Oracle & Open Source" - O'Reilly

View raw message