lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: Large Index and OutOfMemoryError: Map failed
Date Thu, 12 Apr 2012 16:04:09 GMT

On Apr 12, 2012, at 6:07 AM, Michael McCandless wrote:

> Your largest index has 66 segments (690 files) ... biggish but not
> insane.  With 64K maps you should be able to have ~47 searchers open
> on each core.
> 
> Enabling compound file format (not the opposite!) will mean fewer maps
> ... ie should improve this situation.
> 
> I don't understand why Solr defaults to compound file off... that
> seems dangerous.
> 
> Really we need a Solr dev here... to answer "how long is a stale
> searcher kept open".  Is it somehow possible 46 old searchers are
> being left open...?

Probably only if there is a bug. When a new Searcher is opened, any previous Searcher is closed
as soon as there are no more references to it (eg all in flight requests to that Searcher
finish).

> 
> I don't see any other reason why you'd run out of maps.  Hmm, unless
> MMapDirectory didn't think it could safely invoke unmap in your JVM.
> Which exact JVM are you using?  If you can print the
> MMapDirectory.UNMAP_SUPPORTED constant, we'd know for sure.
> 
> Yes, switching away from MMapDir will sidestep the "too many maps"
> issue, however, 1) MMapDir has better perf than NIOFSDir, and 2) if
> there really is a leak here (Solr not closing the old searchers or a
> Lucene bug or something...) then you'll eventually run out of file
> descriptors (ie, same  problem, different manifestation).
> 
> Mike McCandless
> 
> http://blog.mikemccandless.com
> 
> 2012/4/11 Gopal Patwa <gopalpatwa@gmail.com>:
>> 
>> I have not change the mergefactor, it was 10. Compound index file is disable
>> in my config but I read from below post, that some one had similar issue and
>> it was resolved by switching from compound index file format to non-compound
>> index file.
>> 
>> and some folks resolved by "changing lucene code to disable MMapDirectory."
>> Is this best practice to do, if so is this can be done in configuration?
>> 
>> http://lucene.472066.n3.nabble.com/MMapDirectory-failed-to-map-a-23G-compound-index-segment-td3317208.html
>> 
>> I have index document of core1 = 5 million, core2=8million and
>> core3=3million and all index are hosted in single Solr instance
>> 
>> I am going to use Solr for our site StubHub.com, see attached "ls -l" list
>> of index files for all core
>> 
>> SolrConfig.xml:
>> 
>> 
>> 	<indexDefaults>
>> 		<useCompoundFile>false</useCompoundFile>
>> 		<mergeFactor>10</mergeFactor>
>> 		<maxMergeDocs>2147483647</maxMergeDocs>
>> 		<maxFieldLength>10000</maxFieldLength-->
>> 		<ramBufferSizeMB>4096</ramBufferSizeMB>
>> 		<maxThreadStates>10</maxThreadStates>
>> 		<writeLockTimeout>1000</writeLockTimeout>
>> 		<commitLockTimeout>10000</commitLockTimeout>
>> 		<lockType>single</lockType>
>> 		
>> 	    <mergePolicy class="org.apache.lucene.index.TieredMergePolicy">
>> 	      <double name="forceMergeDeletesPctAllowed">0.0</double>
>> 	      <double name="reclaimDeletesWeight">10.0</double>
>> 	    </mergePolicy>
>> 
>> 	    <deletionPolicy class="solr.SolrDeletionPolicy">
>> 	      <str name="keepOptimizedOnly">false</str>
>> 	      <str name="maxCommitsToKeep">0</str>
>> 	    </deletionPolicy>
>> 		
>> 	</indexDefaults>
>> 
>> 
>> 	<updateHandler class="solr.DirectUpdateHandler2">
>> 	    <maxPendingDeletes>1000</maxPendingDeletes>
>> 	     <autoCommit>
>> 	       <maxTime>900000</maxTime>
>> 	       <openSearcher>false</openSearcher>
>> 	     </autoCommit>
>> 	     <autoSoftCommit>
>> 	       <maxTime>${inventory.solr.softcommit.duration:1000}</maxTime>
>> 	     </autoSoftCommit>
>> 	
>> 	</updateHandler>
>> 
>> 
>> Forwarded conversation
>> Subject: Large Index and OutOfMemoryError: Map failed
>> ------------------------
>> 
>> From: Gopal Patwa <gopalpatwa@gmail.com>
>> Date: Fri, Mar 30, 2012 at 10:26 PM
>> To: solr-user@lucene.apache.org
>> 
>> 
>> I need help!!
>> 
>> 
>> 
>> 
>> 
>> I am using Solr 4.0 nightly build with NRT and I often get this error during
>> auto commit "java.lang.OutOfMemoryError: Map failed". I have search this
>> forum and what I found it is related to OS ulimit setting, please se below
>> my ulimit settings. I am not sure what ulimit setting I should have? and we
>> also get "java.net.SocketException: Too many open files" NOT sure how many
>> open file we need to set?
>> 
>> 
>> I have 3 core with index size : core1 - 70GB, Core2 - 50GB and Core3 - 15GB,
>> with Single shard
>> 
>> 
>> We update the index every 5 seconds, soft commit every 1 second and hard
>> commit every 15 minutes
>> 
>> 
>> Environment: Jboss 4.2, JDK 1.6 , CentOS, JVM Heap Size = 24GB
>> 
>> 
>> ulimit:
>> 
>> core file size          (blocks, -c) 0
>> data seg size           (kbytes, -d) unlimited
>> scheduling priority             (-e) 0
>> file size               (blocks, -f) unlimited
>> pending signals                 (-i) 401408
>> max locked memory       (kbytes, -l) 1024
>> max memory size         (kbytes, -m) unlimited
>> open files                      (-n) 1024
>> pipe size            (512 bytes, -p) 8
>> POSIX message queues     (bytes, -q) 819200
>> real-time priority              (-r) 0
>> stack size              (kbytes, -s) 10240
>> cpu time               (seconds, -t) unlimited
>> max user processes              (-u) 401408
>> virtual memory          (kbytes, -v) unlimited
>> file locks                      (-x) unlimited
>> 
>> 
>> 
>> ERROR:
>> 
>> 
>> 
>> 
>> 
>> 2012-03-29 15:14:08,560 [] priority=ERROR app_name= thread=pool-3-thread-1
>> location=CommitTracker line=93 auto commit error...:java.io.IOException: Map
>> failed
>> 	at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
>> 	at
>> org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:293)
>> 	at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:221)
>> 	at
>> org.apache.lucene.codecs.lucene40.Lucene40PostingsReader.<init>(Lucene40PostingsReader.java:58)
>> 	at
>> org.apache.lucene.codecs.lucene40.Lucene40PostingsFormat.fieldsProducer(Lucene40PostingsFormat.java:80)
>> 	at
>> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader$1.visitOneFormat(PerFieldPostingsFormat.java:189)
>> 	at
>> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$VisitPerFieldFile.<init>(PerFieldPostingsFormat.java:280)
>> 	at
>> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader$1.<init>(PerFieldPostingsFormat.java:186)
>> 	at
>> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:186)
>> 	at
>> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:256)
>> 	at
>> org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:108)
>> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:51)
>> 	at
>> org.apache.lucene.index.IndexWriter$ReadersAndLiveDocs.getReader(IndexWriter.java:494)
>> 	at
>> org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:214)
>> 	at
>> org.apache.lucene.index.IndexWriter.applyAllDeletes(IndexWriter.java:2939)
>> 	at
>> org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:2930)
>> 	at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2681)
>> 	at
>> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2804)
>> 	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2786)
>> 	at
>> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:391)
>> 	at org.apache.solr.update.CommitTracker.run(CommitTracker.java:197)
>> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> 	at
>> 
>> ...
>> 
>> [Message clipped]
>> ----------
>> From: Michael McCandless <lucene@mikemccandless.com>
>> Date: Sat, Mar 31, 2012 at 3:15 AM
>> To: solr-user@lucene.apache.org
>> 
>> 
>> It's the virtual memory limit that matters; yours says unlimited below
>> (good!), but, are you certain that's really the limit your Solr
>> process runs with?
>> 
>> On Linux, there is also a per-process map count:
>> 
>>    cat /proc/sys/vm/max_map_count
>> 
>> I think it typically defaults to 65,536 but you should check on your
>> env.  If a process tries to map more than this many regions, you'll
>> hit that exception.
>> 
>> I think you can:
>> 
>>  cat /proc/<pid>/maps | wc
>> 
>> to see how many maps your Solr process currently has... if that is
>> anywhere near the limit then it could be the cause.
>> 
>> Mike McCandless
>> 
>> http://blog.mikemccandless.com
>> 
>> On Sat, Mar 31, 2012 at 1:26 AM, Gopal Patwa <gopalpatwa@gmail.com> wrote:
>>> *I need help!!*
>>> 
>>> *
>>> *
>>> 
>>> *I am using Solr 4.0 nightly build with NRT and I often get this error
>>> during auto commit "**java.lang.OutOfMemoryError:* *Map* *failed". I
>>> have search this forum and what I found it is related to OS ulimit
>>> setting, please se below my ulimit settings. I am not sure what ulimit
>>> setting I should have? and we also get "**java.net.SocketException:*
>>> *Too* *many* *open* *files" NOT sure how many open file we need to
>>> set?*
>>> 
>>> 
>>> I have 3 core with index size : core1 - 70GB, Core2 - 50GB and Core3 -
>>> 15GB, with Single shard
>>> 
>>> *
>>> *
>>> 
>>> *We update the index every 5 seconds, soft commit every 1 second and
>>> hard commit every 15 minutes*
>>> 
>>> *
>>> *
>>> 
>>> *Environment: Jboss 4.2, JDK 1.6 , CentOS, JVM Heap Size = 24GB*
>>> 
>>> *
>>> *
>>> 
>>> ulimit:
>>> 
>>> core file size          (blocks, -c) 0
>>> data seg size           (kbytes, -d) unlimited
>>> scheduling priority             (-e) 0
>>> file size               (blocks, -f) unlimited
>>> pending signals                 (-i) 401408
>>> max locked memory       (kbytes, -l) 1024
>>> max memory size         (kbytes, -m) unlimited
>>> open files                      (-n) 1024
>>> pipe size            (512 bytes, -p) 8
>>> POSIX message queues     (bytes, -q) 819200
>>> real-time priority              (-r) 0
>>> stack size              (kbytes, -s) 10240
>>> cpu time               (seconds, -t) unlimited
>>> max user processes              (-u) 401408
>>> virtual memory          (kbytes, -v) unlimited
>>> file locks                      (-x) unlimited
>>> 
>>> 
>>> *
>>> *
>>> 
>>> *ERROR:*
>>> 
>>> *
>>> *
>>> 
>>> *2012-03-29* *15:14:08*,*560* [] *priority=ERROR* *app_name=*
>>> *thread=pool-3-thread-1* *location=CommitTracker* *line=93* *auto*
>>> *commit* *error...:java.io.IOException:* *Map* *failed*
>>>        *at* *sun.nio.ch.FileChannelImpl.map*(*FileChannelImpl.java:748*)
>>>        *at*
>>> *org.apache.lucene.store.MMapDirectory$MMapIndexInput.*<*init*>(*MMapDirectory.java:293*)
>>>        *at*
>>> *org.apache.lucene.store.MMapDirectory.openInput*(*MMapDirectory.java:221*)
>>>        *at*
>>> *org.apache.lucene.codecs.lucene40.Lucene40PostingsReader.*<*init*>(*Lucene40PostingsReader.java:58*)
>>>        *at*
>>> *org.apache.lucene.codecs.lucene40.Lucene40PostingsFormat.fieldsProducer*(*Lucene40PostingsFormat.java:80*)
>>>        *at*
>>> *org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader$1.visitOneFormat*(*PerFieldPostingsFormat.java:189*)
>>>        *at*
>>> *org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$VisitPerFieldFile.*<*init*>(*PerFieldPostingsFormat.java:280*)
>>>        *at*
>>> *org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader$1.*<*init*>(*PerFieldPostingsFormat.java:186*)
>>>        *at*
>>> *org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.*<*init*>(*PerFieldPostingsFormat.java:186*)
>>>        *at*
>>> *org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer*(*PerFieldPostingsFormat.java:256*)
>>>        *at*
>>> *org.apache.lucene.index.SegmentCoreReaders.*<*init*>(*SegmentCoreReaders.java:108*)
>>>        *at*
>>> *org.apache.lucene.index.SegmentReader.*<*init*>(*SegmentReader.java:51*)
>>>        *at*
>>> *org.apache.lucene.index.IndexWriter$ReadersAndLiveDocs.getReader*(*IndexWriter.java:494*)
>>>        *at*
>>> *org.apache.lucene.index.BufferedDeletesStream.applyDeletes*(*BufferedDeletesStream.java:214*)
>>>        *at*
>>> *org.apache.lucene.index.IndexWriter.applyAllDeletes*(*IndexWriter.java:2939*)
>>>        *at*
>>> *org.apache.lucene.index.IndexWriter.maybeApplyDeletes*(*IndexWriter.java:2930*)
>>>        *at*
>>> *org.apache.lucene.index.IndexWriter.prepareCommit*(*IndexWriter.java:2681*)
>>>        *at*
>>> *org.apache.lucene.index.IndexWriter.commitInternal*(*IndexWriter.java:2804*)
>>>        *at*
>>> *org.apache.lucene.index.IndexWriter.commit*(*IndexWriter.java:2786*)
>>>        *at*
>>> *org.apache.solr.update.DirectUpdateHandler2.commit*(*DirectUpdateHandler2.java:391*)
>>>        *at*
>>> *org.apache.solr.update.CommitTracker.run*(*CommitTracker.java:197*)
>>>        *at*
>>> *java.util.concurrent.Executors$RunnableAdapter.call*(*Executors.java:441*)
>>>        *at*
>>> *java.util.concurrent.FutureTask$Sync.innerRun*(*FutureTask.java:303*)
>>>        *at* *java.util.concurrent.FutureTask.run*(*FutureTask.java:138*)
>>>        *at*
>>> *java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301*(*ScheduledThreadPoolExecutor.java:98*)
>>>        *at*
>>> *java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run*(*ScheduledThreadPoolExecutor.java:206*)
>>>        *at*
>>> *java.util.concurrent.ThreadPoolExecutor$Worker.runTask*(*ThreadPoolExecutor.java:886*)
>>>        *at*
>>> *java.util.concurrent.ThreadPoolExecutor$Worker.run*(*ThreadPoolExecutor.java:908*)
>>>        *at* *java.lang.Thread.run*(*Thread.java:662*)*Caused* *by:*
>>> *java.lang.OutOfMemoryError:* *Map* *failed*
>>>        *at* *sun.nio.ch.FileChannelImpl.map0*(*Native* *Method*)
>>>        *at* *sun.nio.ch.FileChannelImpl.map*(*FileChannelImpl.java:745*)
>>>        *...* *28* *more*
>>> 
>>> *
>>> *
>>> 
>>> *
>>> *
>>> 
>>> *
>>> 
>>> 
>>> SolrConfig.xml:
>>> 
>>> 
>>>        <indexDefaults>
>>>                <useCompoundFile>false</useCompoundFile>
>>>                <mergeFactor>10</mergeFactor>
>>>                <maxMergeDocs>2147483647</maxMergeDocs>
>>>                <maxFieldLength>10000</maxFieldLength-->
>>>                <ramBufferSizeMB>4096</ramBufferSizeMB>
>>>                <maxThreadStates>10</maxThreadStates>
>>>                <writeLockTimeout>1000</writeLockTimeout>
>>>                <commitLockTimeout>10000</commitLockTimeout>
>>>                <lockType>single</lockType>
>>> 
>>>            <mergePolicy class="org.apache.lucene.index.TieredMergePolicy">
>>>              <double name="forceMergeDeletesPctAllowed">0.0</double>
>>>              <double name="reclaimDeletesWeight">10.0</double>
>>>            </mergePolicy>
>>> 
>>>            <deletionPolicy class="solr.SolrDeletionPolicy">
>>>              <str name="keepOptimizedOnly">false</str>
>>>              <str name="maxCommitsToKeep">0</str>
>>>            </deletionPolicy>
>>> 
>>>        </indexDefaults>
>>> 
>>> 
>>>        <updateHandler class="solr.DirectUpdateHandler2">
>>>            <maxPendingDeletes>1000</maxPendingDeletes>
>>>             <autoCommit>
>>>               <maxTime>900000</maxTime>
>>>               <openSearcher>false</openSearcher>
>>>             </autoCommit>
>>>             <autoSoftCommit>
>>> 
>>> <maxTime>${inventory.solr.softcommit.duration:1000}</maxTime>
>>>             </autoSoftCommit>
>>> 
>>>        </updateHandler>
>>> 
>>> 
>>> 
>>> Thanks
>>> Gopal Patwa
>>> *
>> 
>> ----------
>> From: Gopal Patwa <gopalpatwa@gmail.com>
>> Date: Tue, Apr 10, 2012 at 8:35 PM
>> To: solr-user@lucene.apache.org
>> 
>> 
>> Michael, Thanks for response
>> 
>> it was 65K as you mention the default value for "cat
>> /proc/sys/vm/max_map_count" . How we determine what value this should be?
>>  is it number of document during hard commit in my case it is 15 minutes? or
>> it is number of  index file or number of documents we have in all cores.
>> 
>> I have raised the number to 140K but I still get when it reaches to 140K, we
>> have to restart jboss server to free up the map count, sometime OOM error
>> happen during "Error opening new searcher"
>> 
>> is making this number to unlimited is only solution?''
>> 
>> 
>> Error log:
>> 
>> location=CommitTracker line=93 auto commit
>> error...:org.apache.solr.common.SolrException: Error opening new searcher
>> 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1138)
>> 	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1251)
>> 	at
>> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:409)
>> 	at org.apache.solr.update.CommitTracker.run(CommitTracker.java:197)
>> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> 	at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
>> 	at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
>> 	at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> 	at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> 	at java.lang.Thread.run(Thread.java:662)
>> Caused by: java.io.IOException: Map failed
>> 	at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
>> 	at
>> org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:293)
>> 	at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:221)
>> 	at
>> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$VisitPerFieldFile.<init>(PerFieldPostingsFormat.java:262)
>> 	at
>> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$1.<init>(PerFieldPostingsFormat.java:316)
>> 	at
>> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.files(PerFieldPostingsFormat.java:316)
>> 	at org.apache.lucene.codecs.Codec.files(Codec.java:56)
>> 	at org.apache.lucene.index.SegmentInfo.files(SegmentInfo.java:423)
>> 	at org.apache.lucene.index.SegmentInfo.sizeInBytes(SegmentInfo.java:215)
>> 	at
>> org.apache.lucene.index.IndexWriter.prepareFlushedSegment(IndexWriter.java:2220)
>> 	at
>> org.apache.lucene.index.DocumentsWriter.publishFlushedSegment(DocumentsWriter.java:497)
>> 	at
>> org.apache.lucene.index.DocumentsWriter.finishFlush(DocumentsWriter.java:477)
>> 	at
>> org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:201)
>> 	at
>> org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:119)
>> 	at
>> org.apache.lucene.index.DocumentsWriterFlushQueue.tryPurge(DocumentsWriterFlushQueue.java:148)
>> 	at org.apache.lucene.index.
>> 
>> ...
>> 
>> [Message clipped]
>> ----------
>> From: Michael McCandless <lucene@mikemccandless.com>
>> Date: Wed, Apr 11, 2012 at 2:20 AM
>> To: solr-user@lucene.apache.org
>> 
>> 
>> Hi,
>> 
>> 65K is already a very large number and should have been sufficient...
>> 
>> However: have you increased the merge factor?  Doing so increases the
>> open files (maps) required.
>> 
>> Have you disabled compound file format?  (Hmmm: I think Solr does so
>> by default... which is dangerous).  Maybe try enabling compound file
>> format?
>> 
>> Can you "ls -l" your index dir and post the results?
>> 
>> It's also possible Solr isn't closing the old searchers quickly enough
>> ... I don't know the details on when Solr closes old searchers...
>> 
>> Mike McCandless
>> 
>> http://blog.mikemccandless.com
>> 
>> 
>> 
>> On Tue, Apr 10, 2012 at 11:35 PM, Gopal Patwa <gopalpatwa@gmail.com> wrote:
>>> Michael, Thanks for response
>>> 
>>> it was 65K as you mention the default value for "cat
>>> /proc/sys/vm/max_map_count" . How we determine what value this should be?
>>>  is it number of document during hard commit in my case it is 15 minutes?
>>> or it is number of  index file or number of documents we have in all
>>> cores.
>>> 
>>> I have raised the number to 140K but I still get when it reaches to 140K,
>>> we have to restart jboss server to free up the map count, sometime OOM
>>> error happen during "*Error opening new searcher"*
>>> 
>>> is making this number to unlimited is only solution?''
>>> 
>>> 
>>> Error log:
>>> 
>>> *location=CommitTracker line=93 auto commit
>>> error...:org.apache.solr.common.SolrException: Error opening new
>>> searcher
>>>        at
>>> org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1138)
>>>        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1251)
>>>        at
>>> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:409)
>>>        at org.apache.solr.update.CommitTracker.run(CommitTracker.java:197)
>>>        at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>        at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>        at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
>>>        at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
>>>        at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>        at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>        at java.lang.Thread.run(Thread.java:662)Caused by:
>>> java.io.IOException: Map failed
>>>        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
>>>        at
>>> org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:293)
>>>        at
>>> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:221)
>>>        at
>>> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$VisitPerFieldFile.<init>(PerFieldPostingsFormat.java:262)
>>>        at
>>> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$1.<init>(PerFieldPostingsFormat.java:316)
>>>        at
>>> org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.files(PerFieldPostingsFormat.java:316)
>>>        at org.apache.lucene.codecs.Codec.files(Codec.java:56)
>>>        at org.apache.lucene.index.SegmentInfo.files(SegmentInfo.java:423)
>>>        at
>>> org.apache.lucene.index.SegmentInfo.sizeInBytes(SegmentInfo.java:215)
>>>        at
>>> org.apache.lucene.index.IndexWriter.prepareFlushedSegment(IndexWriter.java:2220)
>>>        at
>>> org.apache.lucene.index.DocumentsWriter.publishFlushedSegment(DocumentsWriter.java:497)
>>>        at
>>> org.apache.lucene.index.DocumentsWriter.finishFlush(DocumentsWriter.java:477)
>>>        at
>>> org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:201)
>>>        at
>>> org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:119)
>>>        at
>>> org.apache.lucene.index.DocumentsWriterFlushQueue.tryPurge(DocumentsWriterFlushQueue.java:148)
>>>        at
>>> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:438)
>>>        at
>>> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:553)
>>>        at
>>> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:354)
>>>        at
>>> org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:258)
>>>        at
>>> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:243)
>>>        at
>>> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:250)
>>>        at
>>> org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1091)
>>>        ... 11 moreCaused by: java.lang.OutOfMemoryError: Map failed
>>>        at sun.nio.ch.FileChannelImpl.map0(Native Method)
>>>        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745)*
>>> 
>>> 
>>> 
>>> And one more issue we came across i.e
>> 
>> 

- Mark Miller
lucidimagination.com












Mime
View raw message