lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yang <teddyyyy...@gmail.com>
Subject Re: OutOfMemoryError when opening the index ?
Date Wed, 13 Jun 2012 23:19:54 GMT
ok, found it:

we are using Cloudera CDHu3u, they change the ulimit for child jobs.
but I still don't know how to change their default settings yet


On Wed, Jun 13, 2012 at 2:15 PM, Yang <teddyyyy123@gmail.com> wrote:

> I got the OutOfMemoryError when I tried to open an Lucene index.
>
> it's very weird since this is only seen when I run this inside an Apache
> PIG LATIN script on a particular hadoop cluster of ours,
> and it runs fine on an old cluster; also it runs fine if I run the java
> code outside of PIG.
>
>
> as I understand, OutOfMemoryError is not out of heap, so does this mean
> that mmap() failed?
> I checked the ulimit of the user account running this, it seems to be all
> unlimited
>
>
> any clues?
>
> Thanks
> yang
>
> Caused by: java.io.IOException: Map failed
>         at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
>         at
> org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:265)
>         at
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
>         at
> org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:86)
>         at
> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115)
>         at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:93)
>         at
> org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:113)
>         at
> org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:29)
>         at
> org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:81)
>         at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:754)
>         at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:75)
>         at org.apache.lucene.index.IndexReader.open(IndexReader.java:462)
>         at org.apache.lucene.index.IndexReader.open(IndexReader.java:308)
>         at
> cgm.places.matcher.ListingSearcher.createIndex(ListingSearcher.java:45)
>         at
> cgm.places.matcher.ListingSearcher.<init>(ListingSearcher.java:38)
>         at
> cgm.places.matcher.SearchListingUdf.<init>(SearchListingUdf.java:34)
>         ... 83 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>         at sun.nio.ch.FileChannelImpl.map0(Native Method)
>         at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745)
>         ... 98 more
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message