lucy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick D." <ndw...@globaldataguard.com>
Subject Re: [lucy-user] 32 bit CentOS Indexing Question
Date Thu, 13 Feb 2014 15:40:11 GMT
Nick Wellnhofer wrote
> On Jan 29, 2014, at 02:59 , Marvin Humphrey &lt;

> marvin@

> &gt; wrote:
> 
>> On Tue, Jan 28, 2014 at 11:26 AM, Nick D. &lt;

> ndwyer@

> &gt; wrote:
>>> Why do I get a memory allocation error on a 32 bit OS and not a 64 bit
>>> OS?
>> 
>> It's probably a known architectural flaw in SortWriter which makes it
>> consume
>> too much RAM.
> 
> 
> It would be interesting to know whether they make a difference in Nick
> D.’s case. If they solve his problem, we should consider backporting the
> fix to the 0.3 branch.
> 
> Nick W.

It did not fix the issue but committing every 50k records or so (syslog
style records) fixed the issue and sped up indexing a bit. I was able to
install the SortWriter branch and unfortunately the set_mem_threshold did
not speed up indexing (possibly I/O device is the bottleneck).




--
View this message in context: http://lucene.472066.n3.nabble.com/32-bit-CentOS-Indexing-Question-tp4114036p4117201.html
Sent from the lucy-user mailing list archive at Nabble.com.

Mime
View raw message