Return-Path: 
 <solr-dev-return-22297-apmail-lucene-solr-dev-archive=lucene.apache.org@lucene.apache.org>
Delivered-To: apmail-lucene-solr-dev-archive@minotaur.apache.org
Received: (qmail 53962 invoked from network); 4 Dec 2009 21:31:45 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
  by minotaur.apache.org with SMTP; 4 Dec 2009 21:31:45 -0000
Received: (qmail 72765 invoked by uid 500); 4 Dec 2009 21:31:45 -0000
Delivered-To: apmail-lucene-solr-dev-archive@lucene.apache.org
Received: (qmail 72700 invoked by uid 500); 4 Dec 2009 21:31:44 -0000
Mailing-List: contact solr-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:solr-dev-help@lucene.apache.org>
List-Unsubscribe: <mailto:solr-dev-unsubscribe@lucene.apache.org>
List-Post: <mailto:solr-dev@lucene.apache.org>
List-Id: <solr-dev.lucene.apache.org>
Reply-To: solr-dev@lucene.apache.org
Delivered-To: mailing list solr-dev@lucene.apache.org
Received: (qmail 72662 invoked by uid 99); 4 Dec 2009 21:31:44 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Dec 2009 21:31:44 +0000
X-ASF-Spam-Status: No, hits=-10.5 required=5.0
	tests=AWL,BAYES_00,RCVD_IN_DNSWL_HI
X-Spam-Check-By: apache.org
Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Dec 2009 21:31:42 +0000
Received: from brutus (localhost [127.0.0.1])
	by brutus.apache.org (Postfix) with ESMTP id 4C203234C052
	for <solr-dev@lucene.apache.org>; Fri,  4 Dec 2009 13:31:22 -0800 (PST)
Message-ID: <359514854.1259962282310.JavaMail.jira@brutus>
Date: Fri, 4 Dec 2009 21:31:22 +0000 (UTC)
From: "Yonik Seeley (JIRA)" <jira@apache.org>
To: solr-dev@lucene.apache.org
Subject: [jira] Commented: (SOLR-1623) Solr hangs (often throwing
 java.lang.OutOfMemoryError: PermGen space) when indexing many different
 field names
In-Reply-To: <1454624508.1259959400812.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


    [ https://issues.apache.org/jira/browse/SOLR-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786175#action_12786175 ] 

Yonik Seeley commented on SOLR-1623:
------------------------------------

bq. Whats odd is that he has it marked as affects 1.4 as well - but that doesn't intern to perm gen anymore?

The default StringHelper.intern() from Lucene is just a cache - String.intern() is still called.

> Solr hangs (often throwing java.lang.OutOfMemoryError: PermGen space) when indexing many different field names
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-1623
>                 URL: https://issues.apache.org/jira/browse/SOLR-1623
>             Project: Solr
>          Issue Type: Bug
>          Components: update
>    Affects Versions: 1.3, 1.4
>         Environment: Tomcat Version         JVM Version                      JVM Vendor                    OS Name OS Version        OS Architecture 
> Apache Tomcat/6.0   snapshot 1.6.0_13-b03 Sun Microsystems Inc. Linux         2.6.18-164.el5  amd64 
> and/or
> Tomcat Version                JVM Version         JVM Vendor                        OS Name               OS Version    OS Architecture 
> Apache Tomcat/6.0.18   1.6.0_12-b04        Sun Microsystems Inc.     Windows 2003     5.2                   amd64 
>            Reporter: Laurent Chavet
>            Priority: Critical
>
> With the following fields in schema.xml:
>  <fields>
>    <field name="id" type="sint" indexed="true" stored="true" required="true" /> 
>     <dynamicField name="weight_*"  type="sint"    indexed="true"  stored="true"/>
> </fields>
> Run the following code:
> import java.util.ArrayList;
> import java.util.List;
> import org.apache.solr.client.solrj.SolrServer;
> import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer;
> import org.apache.solr.common.SolrInputDocument;
>     public static void main(String[] args) throws Exception {
>         SolrServer server;
>         try {
>             server = new CommonsHttpSolrServer(args[0]);
>         } catch (Exception e) {
>             System.err.println("can't creater server using: " + args[0] + "  " + e.getMessage());
>             throw e;
>         }
>         for (int i = 0; i < 1000; i++) {
>             List<SolrInputDocument> batchedDocs = new ArrayList<SolrInputDocument>();
>             for (int j = 0; j < 1000; j++) {
>                 SolrInputDocument doc = new SolrInputDocument();
>                 doc.addField("id", i * 1000 + j);
>                 // hangs after 30 to 50 batches
>                 doc.addField("weight_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + Integer.toString(i) + "_" + Integer.toString(j), i * 1000 + j);
>                 // hangs after about 200 batches
>                 //doc.addField("weight_" + Integer.toString(i) + "_" + Integer.toString(j), i * 1000 + j);
>                 batchedDocs.add(doc);
>             }
>             try {
>                 server.add(batchedDocs, true);
>                 System.err.println("Done with batch=" + i);
>                 // server.commit(); //doesn't change anything
>             } catch (Exception e) {
>                 System.err.println("batchId=" + i + " bad batch: " + e.getMessage());
>                 throw e;
>             }
>         }
>     }
> And soon the client (sometime throws) and solr will freeze. sometime you can see: java.lang.OutOfMemoryError: PermGen space in the server logs

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.