lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitry Serebrennikov <dmit...@earthlink.net>
Subject Re: Compound file index issue
Date Thu, 15 Jan 2004 00:01:38 GMT
Hello Bruce,

I've seen a couple of other people reporting similar issues when 
switching to the latest 1.3 tree. However, usually the problems come 
from lack of synchronization of multiple threads doing reads and writes 
against the index. But it's definetely possible that the compound index 
format makes this issue more visible or that it has its own bugs. In my 
testing and in the initial testing of the Lucene dev team when we were 
working on the compound index format we didn't see any such issues, but 
of course that could just mean that we didn't look in the right place.

Anyway, it would be extremely helpful if you could produce a 
self-contained program that demonstrates this behavior. I know that this 
can be hard to impossible with concurrency-related issues, but still.

Also, what OS is this running under? This issue is related to filesystem 
locking behavior which is different between Unix and Windows.
Does this happen every time or does it happen only under heavy load, etc.?
What is the system doing when this occurs (i.e. indexing and searching 
at the same time or just searching etc)?


Thanks.
Dmitry.


Bruce Ritchie wrote:

> We've started using the new compound indexing functionality available 
> in lucene 1.3 final however we've been seeing an issue that I haven't 
> quite resolved and was hoping that people on this list might be able 
> to lend help on.
>
> The stack trace of the issue is:
>
> 2004.01.13 18:15 
> [com.jivesoftware.forum.database.DbQuery.executeQuery(DbQuery.java:402)]
> java.io.FileNotFoundException: /var/search/BfUfO9s1/_qb5.fnm (No such 
> file or directory)
>         at java.io.RandomAccessFile.open(Native Method)
>         at java.io.RandomAccessFile.<init>(RandomAccessFile.java:204)
>         at 
> org.apache.lucene.store.FSInputStream$Descriptor.<init>(FSDirectory.java:392)
>         at 
> org.apache.lucene.store.FSInputStream.<init>(FSDirectory.java:421)
>         at 
> org.apache.lucene.store.FSDirectory.openFile(FSDirectory.java:294)
>         at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:80)
>         at 
> org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:138)
>         at 
> org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:120)
>         at 
> org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:122)
>         at org.apache.lucene.store.Lock$With.run(Lock.java:148)
>         at org.apache.lucene.index.IndexReader.open(IndexReader.java:111)
>         at 
> com.jivesoftware.forum.database.DbSearchManager.getSearcher(DbSearchManager.java:897)
>         ...
>
> The error seems obvious enough from a quick glance of the above code - 
> the file was been deleted. However, since we are using compound 
> indexing there should be a /var/search/BfUfO9s1/_qb5.cfs file, however 
> that file does not exist. The only files in the search directory are:
>
> -bash-2.05b$ ls -l
> total 6908
> -rw-r--r--    1 resin    js              4 Jan 13 18:14 deletable
> -rw-r--r--    1 resin    js        7029055 Jan 13 10:03 _qau.cfs
> -rw-r--r--    1 resin    js              9 Jan 13 18:14 _qb5.del
> -rw-r--r--    1 resin    js          16060 Jan 13 18:14 _qb7.cfs
> -rw-r--r--    1 resin    js             34 Jan 13 18:14 segments
>
>
> From what I can tell it seems to me that the segments are somehow 
> getting out of sync with what is actually on the filesystem somehow.
>
> Any suggestions are where to start looking for the cause of this bug?
>
>
> Regards,
>
> Bruce Ritchie




---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message