lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <>
Subject RE: Custom Directory and IndexInput
Date Tue, 21 Aug 2012 06:31:01 GMT


It is required that every thread uses its own IndexInput. This is why e.g. every search calls
IndexInput.clone() several times while executing a search query. Because of that there are
explicietly no thread safety requirements in IndexInput, but clone() must be implemented as
a cheap method, cloning the IndexInput and keeping the state. It is not required to recreate
file descriptors or like that, but it must make sure that the new IndexInput can be used independent,
although on same file/same file descriptor like another one.





Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen <> 



From: Danil Ε’ORIN [] 
Sent: Tuesday, August 21, 2012 8:22 AM
Subject: Custom Directory and IndexInput


I'm trying to build a custom directory implementation, actually the directory itself just
fallback to one of the existing directory implementations, so it's actually more about IndexInput/IndexOutput.

I have some concerns about my implementation, especially related to multithreading:

*	IndexOutput writeByte/writeBytes should be no problem since only one thread will write to
specific output at any time
*	IndexInput on the other hand could be called from multiple threads
*	most implementations of IndexInput (including mine) keep some internal state like current
position in the file/buffer/etc
*	if one of the threads is calling readShort()/readLong()/readVInt() (default implementations
from DataInput) while some other thread is calling readByte() bad stuff will happen
*	still in most existing implementations I don't see any synchronization code on reads
*	is this problem solved somewhere on a higher level, like IndexReader/IndexSearcher, so I
don't have to worry about it?
*	cause I'm really not sure how to solve it without forcing synchronized on all read methods
and I'd really hate that.

How this issue is solved in Lucene? what should I do to make sure that my implementation doing
right thing?


Currently I'm on Lucene 4.0.0 BETA


View raw message