I'm trying to build a custom directory implementation, actually the directory itself just fallback to one of the existing directory implementations, so it's actually more about IndexInput/IndexOutput.
I have some concerns about my implementation, especially related to multithreading:
How this issue is solved in Lucene? what should I do to make sure that my implementation doing right thing?
- IndexOutput writeByte/writeBytes should be no problem since only one thread will write to specific output at any time
- IndexInput on the other hand could be called from multiple threads
- most implementations of IndexInput (including mine) keep some internal state like current position in the file/buffer/etc
- if one of the threads is calling readShort()/readLong()/readVInt() (default implementations from DataInput) while some other thread is calling readByte() bad stuff will happen
- still in most existing implementations I don't see any synchronization code on reads
- is this problem solved somewhere on a higher level, like IndexReader/IndexSearcher, so I don't have to worry about it?
- cause I'm really not sure how to solve it without forcing synchronized on all read methods and I'd really hate that.
Currently I'm on Lucene 4.0.0 BETA