Return-Path: X-Original-To: apmail-lucene-java-user-archive@www.apache.org Delivered-To: apmail-lucene-java-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 758739702 for ; Wed, 13 Jun 2012 21:16:00 +0000 (UTC) Received: (qmail 44082 invoked by uid 500); 13 Jun 2012 21:15:58 -0000 Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org Received: (qmail 43968 invoked by uid 500); 13 Jun 2012 21:15:58 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 43955 invoked by uid 99); 13 Jun 2012 21:15:57 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Jun 2012 21:15:57 +0000 X-ASF-Spam-Status: No, hits=3.4 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FSL_RCVD_USER,HK_RANDOM_ENVFROM,HK_RANDOM_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of teddyyyy123@gmail.com designates 209.85.214.176 as permitted sender) Received: from [209.85.214.176] (HELO mail-ob0-f176.google.com) (209.85.214.176) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Jun 2012 21:15:49 +0000 Received: by obbef5 with SMTP id ef5so1781722obb.35 for ; Wed, 13 Jun 2012 14:15:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=85cU840rPPCH2wNIEUvRRWsw7lmyaGymudz2A82rMMg=; b=zxvdlp2gyyXDWNdnI2Rh1XKmKloigjMn0ot5R/2uB7nc2R8zMbmxWNxfH2W2gzRQIY wH+Ys05J2rHpmH7eT8Tdn1ovm/cPIGLY9JWctxUr+iGTm9vC4IxO76SofL78e7BaHROh bEhtLE/Bew5n2tdTiMnUcpxZymlf6zQRUIP69uW6sDbz0LPrxK2mUqkSBWP5x2ck2sKb 4gUpJA3BD7QhdCFAbGPwsQBnDGJWqDe5iOaw6Eiiii4I6ijTqQFcUnayIv6TptWAcRjl rge58NlHIjRcLSJm7q+OVQGFENLc4UPvCx+rddYY1wAoX+cMDVtuysrgvsSBXzHIfPG7 q6ig== Received: by 10.182.18.164 with SMTP id x4mr25992583obd.15.1339622128984; Wed, 13 Jun 2012 14:15:28 -0700 (PDT) MIME-Version: 1.0 Received: by 10.114.37.130 with HTTP; Wed, 13 Jun 2012 14:15:08 -0700 (PDT) From: Yang Date: Wed, 13 Jun 2012 14:15:08 -0700 Message-ID: Subject: OutOfMemoryError when opening the index ? To: java-user@lucene.apache.org Content-Type: multipart/alternative; boundary=f46d043891a521a3fe04c2611551 --f46d043891a521a3fe04c2611551 Content-Type: text/plain; charset=ISO-8859-1 I got the OutOfMemoryError when I tried to open an Lucene index. it's very weird since this is only seen when I run this inside an Apache PIG LATIN script on a particular hadoop cluster of ours, and it runs fine on an old cluster; also it runs fine if I run the java code outside of PIG. as I understand, OutOfMemoryError is not out of heap, so does this mean that mmap() failed? I checked the ulimit of the user account running this, it seems to be all unlimited any clues? Thanks yang Caused by: java.io.IOException: Map failed at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748) at org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:265) at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216) at org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:86) at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115) at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:93) at org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:113) at org.apache.lucene.index.ReadOnlyDirectoryReader.(ReadOnlyDirectoryReader.java:29) at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:81) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:754) at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:75) at org.apache.lucene.index.IndexReader.open(IndexReader.java:462) at org.apache.lucene.index.IndexReader.open(IndexReader.java:308) at cgm.places.matcher.ListingSearcher.createIndex(ListingSearcher.java:45) at cgm.places.matcher.ListingSearcher.(ListingSearcher.java:38) at cgm.places.matcher.SearchListingUdf.(SearchListingUdf.java:34) ... 83 more Caused by: java.lang.OutOfMemoryError: Map failed at sun.nio.ch.FileChannelImpl.map0(Native Method) at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745) ... 98 more --f46d043891a521a3fe04c2611551--