Return-Path: 
 <core-user-return-4797-apmail-hadoop-core-user-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-core-user-archive@www.apache.org
Received: (qmail 65380 invoked from network); 25 Feb 2008 07:56:41 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2)
  by minotaur.apache.org with SMTP; 25 Feb 2008 07:56:41 -0000
Received: (qmail 14321 invoked by uid 500); 25 Feb 2008 07:56:34 -0000
Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org
Received: (qmail 14142 invoked by uid 500); 25 Feb 2008 07:56:33 -0000
Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:core-user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:core-user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:core-user@hadoop.apache.org>
List-Id: <core-user.hadoop.apache.org>
Reply-To: core-user@hadoop.apache.org
Delivered-To: mailing list core-user@hadoop.apache.org
Received: (qmail 14133 invoked by uid 99); 25 Feb 2008 07:56:33 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 24 Feb 2008 23:56:33 -0800
X-ASF-Spam-Status: No, hits=-0.0 required=10.0
	tests=SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: local policy)
Received: from [216.109.112.28] (HELO mrout2-b.corp.dcn.yahoo.com)
 (216.109.112.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 25 Feb 2008 07:55:44 +0000
Received: from sevenshade-lr.eglbp.corp.yahoo.com
 (sevenshade-lr.eglbp.corp.yahoo.com [10.66.74.35])
	by mrout2-b.corp.dcn.yahoo.com (8.13.6/8.13.6/y.out) with ESMTP id
 m1P7ttAr043547;
	Sun, 24 Feb 2008 23:55:56 -0800 (PST)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=date:from:x-x-sender:to:cc:subject:in-reply-to:message-id:
	references:mime-version:content-type;
	b=RX/n8p0C12DB3dkARqCyEzZOfjbkyzhm93n7H2Yd9lzhpQ5JgE1pighR/jhep/fr
Date: Mon, 25 Feb 2008 13:27:06 +0530 (IST)
From: Amar Kamat <amarrk@yahoo-inc.com>
X-X-Sender: amarrk@sevenshade-lr.eglbp.corp.yahoo.com
To: ma qiang <maqiang1984@gmail.com>
cc: core-user@hadoop.apache.org
Subject: Re: about the length of Text
In-Reply-To: <2af70f430802242348i22c446cby859c3cc4f793c203@mail.gmail.com>
Message-ID: 
 <Pine.LNX.4.64.0802251322290.5655@sevenshade-lr.eglbp.corp.yahoo.com>
References: <2af70f430802242348i22c446cby859c3cc4f793c203@mail.gmail.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
X-Virus-Checked: Checked by ClamAV on apache.org

What is the heap size (see mapred.child.java.opts in your job.xml)? By the 
way what is the job configuration i.e what is the job type (sort etc), number of nodes 
etc. Can you provide more details? Anyways try increasing it which you can 
do it using
hadoop jar jar-file.jar program -Dmapred.child.java.opts=-Xmx512m <args>

Amar
On Mon, 25 Feb 2008, ma qiang wrote:

> Hi all;
>    A error happen in my program and system print as below:
>        java.lang.OutOfMemoryError: Java heap space
> 	at java.util.Arrays.copyOf(Arrays.java:2786)
> 	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
> 	at java.io.DataOutputStream.write(DataOutputStream.java:90)
> 	at org.apache.hadoop.io.Text.write(Text.java:243)
> 	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:349)
> 	at MyTest.map(UserMinHashMap.java:43)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:192)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1787)
>
> and I think the error happen here: output.collect(new
> Text(testKeyString),new Text(testValueString));
> I'm not sure why? I guess the reason is the length of testValueString
> is too large to construct Text or write into HD.   Who can tell why?
>
> Thanks very much
> Best Wishes!
>