lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Danil ŢORIN <torin...@gmail.com>
Subject Re: Custom Directory and IndexInput
Date Tue, 21 Aug 2012 08:15:42 GMT
Is there a way to force run full lucene & solr test suite against my new
Directory implementation?

Just to make sure everything is covered :)


On Tue, Aug 21, 2012 at 10:55 AM, Danil ŢORIN <torindan@gmail.com> wrote:

> My bad, few casts solved the issue.
>
>
> On Tue, Aug 21, 2012 at 10:46 AM, Danil ŢORIN <torindan@gmail.com> wrote:
>
>> One tricky thing, I'm doing a lot of delegation, I can't call "clone()"
>> on the DataInput I'm delegating to.
>> (unfortunately in my case it's delegation, not inheritance)
>>
>> The fact that the delegate implements Clonable, doesn't help, the clone()
>> method is still protected :(.
>> Any idea how to circumvent this?
>>
>>
>> On Tue, Aug 21, 2012 at 10:37 AM, Uwe Schindler <uwe@thetaphi.de> wrote:
>>
>>> Thanks for „reporting“ lack of documentation, I will shortly commit some
>>> additional specs to IndexInput’s javadocs!****
>>>
>>> ** **
>>>
>>> Thanks,****
>>>
>>> Uwe****
>>>
>>> ** **
>>>
>>> -----****
>>>
>>> Uwe Schindler****
>>>
>>> H.-H.-Meier-Allee 63, D-28213 Bremen****
>>>
>>> http://www.thetaphi.de****
>>>
>>> eMail: uwe@thetaphi.de****
>>>
>>> ** **
>>>
>>> *From:* Danil ŢORIN [mailto:torindan@gmail.com]
>>> *Sent:* Tuesday, August 21, 2012 8:59 AM
>>> *To:* dev@lucene.apache.org
>>> *Subject:* Re: Custom Directory and IndexInput****
>>>
>>> ** **
>>>
>>> Thanks, that's exactly the response I was looking for.****
>>>
>>> On Tue, Aug 21, 2012 at 9:31 AM, Uwe Schindler <uwe@thetaphi.de> wrote:*
>>> ***
>>>
>>> Hi,****
>>>
>>>  ****
>>>
>>> It is required that every thread uses its own IndexInput. This is why
>>> e.g. every search calls IndexInput.clone() several times while executing a
>>> search query. Because of that there are explicietly no thread safety
>>> requirements in IndexInput, but clone() must be implemented as a cheap
>>> method, cloning the IndexInput and keeping the state. It is not required to
>>> recreate file descriptors or like that, but it must make sure that the new
>>> IndexInput can be used independent, although on same file/same file
>>> descriptor like another one.****
>>>
>>>  ****
>>>
>>> Uwe****
>>>
>>>  ****
>>>
>>> -----****
>>>
>>> Uwe Schindler****
>>>
>>> H.-H.-Meier-Allee 63, D-28213 Bremen****
>>>
>>> http://www.thetaphi.de****
>>>
>>> eMail: uwe@thetaphi.de****
>>>
>>>  ****
>>>
>>> *From:* Danil ŢORIN [mailto:torindan@gmail.com]
>>> *Sent:* Tuesday, August 21, 2012 8:22 AM
>>> *To:* java-dev@lucene.apache.org
>>> *Subject:* Custom Directory and IndexInput****
>>>
>>>  ****
>>>
>>>
>>> I'm trying to build a custom directory implementation, actually the
>>> directory itself just fallback to one of the existing directory
>>> implementations, so it's actually more about IndexInput/IndexOutput.
>>>
>>> I have some concerns about my implementation, especially related to
>>> multithreading:****
>>>
>>>    - IndexOutput writeByte/writeBytes should be no problem since only
>>>    one thread will write to specific output at any time****
>>>    - IndexInput on the other hand could be called from multiple threads*
>>>    ***
>>>    - most implementations of IndexInput (including mine) keep some
>>>    internal state like current position in the file/buffer/etc****
>>>    - if one of the threads is calling readShort()/readLong()/readVInt()
>>>    (default implementations from DataInput) while some other thread is calling
>>>    readByte() bad stuff will happen****
>>>    - still in most existing implementations I don't see any
>>>    synchronization code on reads****
>>>    - is this problem solved somewhere on a higher level, like
>>>    IndexReader/IndexSearcher, so I don't have to worry about it?****
>>>    - cause I'm really not sure how to solve it without forcing
>>>    synchronized on all read methods and I'd really hate that.****
>>>
>>> How this issue is solved in Lucene? what should I do to make sure that
>>> my implementation doing right thing?****
>>>
>>>  ****
>>>
>>> Currently I'm on Lucene 4.0.0 BETA****
>>>
>>>  ****
>>>
>>> ** **
>>>
>>
>>
>

Mime
View raw message