Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of erickerickson@gmail.com
 designates 209.85.216.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=WkJIY1NOK7JK49tWFUxiQkz6KLaC617kLNeOvNKzXA8NOakbUUqtwqPkg1txjNl6wp
         bHKXU4H+pYorrV3qBPvxQq1z1mc2RudcbO4tw1c3ou6T5OigjoxakYkXx7GCt/Hwar/q
         amCJZd7muPHieQFspvfMdyMkyfbXBZLPiwZvU=
MIME-Version: 1.0
In-Reply-To: <20110404081325.yar7bwg7qcg0ggsg@webmail.conectavoce.com>
References: <20110404081325.yar7bwg7qcg0ggsg@webmail.conectavoce.com>
Date: Mon, 4 Apr 2011 07:38:26 -0400
Message-ID: <BANLkTin88+95Y-GLOcVx-q6bJ1wTOYPHyQ@mail.gmail.com>
Subject: Re: OutOfMemoryError with FSDirectory
From: Erick Erickson <erickerickson@gmail.com>
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=0016e64ba0f0ab79f404a01632c1

--0016e64ba0f0ab79f404a01632c1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

FSDirectory will, indeed, store the index on disk. However,
when *using* that index, lots of stuff happens. Specifically:
When indexing, there is a buffer that accumulates documents
until it's flushed to disk. Are you indexing?

When searching (and this is the more important part), various
caches are used to speed up searches. A substantial part of
your index may be held in memory at various points.

70M just isn't very much memory, I think you'll have to get
more if at all possible.

Best
Erick

On Mon, Apr 4, 2011 at 7:13 AM, Claudio <temp@conectavoce.com> wrote:

> Hi,
>
> I am using Lucene 2.9.4 with FSDirectory.
> My index has 80 thousand documents (each document has 12 fields).
> My jvm has 70Mb of RAM memory (limited by my hosting).
> I am getting various OutOfMemoryError.
> I ran jmap and I got:
>
> num       #instances    #bytes    Class description
> -------------------------------------------------------------------------=
-
> 1:        275926    32074080    char[]
> 2:        657724    31570752
>  org.apache.lucene.index.FreqProxTermsWriter$PostingList
> 3:        200    16094896    org.apache.lucene.index.RawPostingList[]
>
> The Lucene is consuming 47Mb of RAM memory.
> Why there is this excessive memory consumption in FSDirectory?
> Is it correct? I'm using FSDirectory because I haven=B4t a big amount of =
RAM
> memory.
>
> Thanks.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

--0016e64ba0f0ab79f404a01632c1--