lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: Deduplication patch not working in nightly build
Date Fri, 09 Jan 2009 17:50:01 GMT
Your basically writing segments more often now, and somehow avoiding a 
longer merge I think. Also, likely, deduplication is probably adding 
enough extra data to your index to hit a sweet spot where a merge is too 
long. Or something to that effect - MySql is especially sensitive to 
timeouts when doing a select * on a huge db in my testing. I didnt 
understand your answer on the autocommit - I take it you are using it? 
Or no?

All a guess, but it def points to a merge taking a bit long and causing 
a timeout. I think you can relax the MySql timeout settings if that is it.

I'd like to get to the bottom of this as well, so any other info you can 
provide would be great.

- Mark

Marc Sturlese wrote:
> Hey Shalin,
>
> In the begining (when the error was appearing) i had 
> <ramBufferSizeMB>32</ramBufferSizeMB>
> and no maxBufferedDocs set
>
> Now I have:
> <ramBufferSizeMB>32</ramBufferSizeMB>
> <maxBufferedDocs>50</maxBufferedDocs>
>
> I think taht setting maxBufferedDocs to 50 I am forcing more disk writting
> than I would like... but at least it works fine (but a bit slower,opiously).
>
> I keep saying that the most weird thing is that I don't have that problem
> using solr1.3, just with the nightly...
>
> Even that it's good that it works well now, would be great if someone can
> give me an explanation why this is happening
>  
>
>
> Shalin Shekhar Mangar wrote:
>   
>> On Fri, Jan 9, 2009 at 9:23 PM, Marc Sturlese
>> <marc.sturlese@gmail.com>wrote:
>>
>>     
>>> hey there,
>>> I hadn't autoCommit set to true but I have it sorted!!!!! The error
>>> stopped
>>> appearing after setting the property maxBufferedDocs in solrconfig.xml. I
>>> can't exactly undersand why but it just worked.
>>> Anyway, maxBufferedDocs is deprecaded, would ramBufferSizeMB do the same?
>>>
>>>
>>>       
>> What I find strange is this line in the exception:
>> "Last packet sent to the server was 202481 ms ago."
>>
>> Something took very very long to complete and the connection got closed by
>> the time the next row was fetched from the opened resultset.
>>
>> Just curious, what was the previous value of maxBufferedDocs and what did
>> you change it to?
>>
>>
>>     
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21374908.html
>>> Sent from the Solr - User mailing list archive at Nabble.com.
>>>
>>>
>>>       
>> -- 
>> Regards,
>> Shalin Shekhar Mangar.
>>
>>
>>     
>
>   


Mime
View raw message