lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarrod Cuzens (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-140) docs out of order
Date Mon, 16 Jan 2006 17:45:21 GMT
    [ http://issues.apache.org/jira/browse/LUCENE-140?page=comments#action_12362866 ] 

Jarrod Cuzens commented on LUCENE-140:
--------------------------------------

Hi Doug,

Thanks for your suggestion (and great work on Lucene!). I looked in the logs and could not
find any indication of corruption. In addition we actually have the system running in a cluster
where each node in the cluster has its own lucene search index. We had the issue on a different
machine before we moved to the cluster and we now see it on both machines in our cluster :(.
Next time it occurs I will get the indexes and try to post them here if you think that would
be helpful.

Just some additional info: 
1)  We are running SUSE 10 Linux.
2)  We run two quartz jobs: One job runs every 2 minutes and updates lucene with changes from
the db if necessary. The other job runs at 3:30AM in the morning and does full index rebuilds.
We use the LuceneIndexAccess api when working with the IndexReader/IndexWriter. Only one thread
should modify the index at any time although it is possible that the full rebuild job and
the update job execute concurrently they shouldn't both modify the index due to the LuceneIndexAccess
API's synchronization.
3) Currently when doing searches we create a new IndexSearcher everytime a search is to be
performed; We do not use the LuceneIndexAccess API.

When I refer to the LuceneIndexAccess API I am referring to the contribution by Mark Schreiber:
http://www.blizzy.de/lucene/lucene-indexaccess-0.1.0.zip

Thanks for any help!


> docs out of order
> -----------------
>
>          Key: LUCENE-140
>          URL: http://issues.apache.org/jira/browse/LUCENE-140
>      Project: Lucene - Java
>         Type: Bug
>   Components: Index
>     Versions: unspecified
>  Environment: Operating System: Linux
> Platform: PC
>     Reporter: legez
>     Assignee: Lucene Developers
>  Attachments: bug23650.txt
>
> Hello,
>   I can not find out, why (and what) it is happening all the time. I got an
> exception:
> java.lang.IllegalStateException: docs out of order
>         at
> org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:219)
>         at
> org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:191)
>         at
> org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:172)
>         at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:135)
>         at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:88)
>         at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:341)
>         at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:250)
>         at Optimize.main(Optimize.java:29)
> It happens either in 1.2 and 1.3rc1 (anyway what happened to it? I can not find
> it neither in download nor in version list in this form). Everything seems OK. I
> can search through index, but I can not optimize it. Even worse after this
> exception every time I add new documents and close IndexWriter new segments is
> created! I think it has all documents added before, because of its size.
> My index is quite big: 500.000 docs, about 5gb of index directory.
> It is _repeatable_. I drop index, reindex everything. Afterwards I add a few
> docs, try to optimize and receive above exception.
> My documents' structure is:
>   static Document indexIt(String id_strony, Reader reader, String data_wydania,
> String id_wydania, String id_gazety, String data_wstawienia)
> {
>     Document doc = new Document();
>     doc.add(Field.Keyword("id", id_strony ));
>     doc.add(Field.Keyword("data_wydania", data_wydania));
>     doc.add(Field.Keyword("id_wydania", id_wydania));
>     doc.add(Field.Text("id_gazety", id_gazety));
>     doc.add(Field.Keyword("data_wstawienia", data_wstawienia));
>     doc.add(Field.Text("tresc", reader));
>     return doc;
> }
> Sincerely,
> legez

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message