Return-Path: Delivered-To: apmail-hadoop-core-user-archive@www.apache.org Received: (qmail 6260 invoked from network); 27 Jun 2008 23:26:51 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 27 Jun 2008 23:26:51 -0000 Received: (qmail 19756 invoked by uid 500); 27 Jun 2008 23:26:47 -0000 Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org Received: (qmail 19727 invoked by uid 500); 27 Jun 2008 23:26:47 -0000 Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-user@hadoop.apache.org Delivered-To: mailing list core-user@hadoop.apache.org Received: (qmail 19715 invoked by uid 99); 27 Jun 2008 23:26:47 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Jun 2008 16:26:47 -0700 X-ASF-Spam-Status: No, hits=-4.0 required=10.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of mbellamy@apple.com designates 17.254.13.23 as permitted sender) Received: from [17.254.13.23] (HELO mail-out4.apple.com) (17.254.13.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Jun 2008 23:25:55 +0000 Received: from relay14.apple.com (relay14.apple.com [17.128.113.52]) by mail-out4.apple.com (Postfix) with ESMTP id 4580D323C24E for ; Fri, 27 Jun 2008 16:26:15 -0700 (PDT) Received: from relay14.apple.com (unknown [127.0.0.1]) by relay14.apple.com (Symantec Mail Security) with ESMTP id 2EAAC2804C for ; Fri, 27 Jun 2008 16:26:15 -0700 (PDT) X-AuditID: 11807134-aa201bb000000ece-05-48657717b874 Received: from da0803a-dhcp153.apple.com (da0803a-dhcp153.apple.com [17.228.35.153]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by relay14.apple.com (Apple SCV relay) with ESMTP id 111492802F for ; Fri, 27 Jun 2008 16:26:15 -0700 (PDT) Message-Id: <8C81447E-93D4-42AE-85A4-3C7BB23885A8@apple.com> From: Mori Bellamy To: core-user@hadoop.apache.org Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v919.2) Subject: hadoop map reduce: additional heap space? Date: Fri, 27 Jun 2008 16:26:14 -0700 X-Mailer: Apple Mail (2.919.2) X-Brightmail-Tracker: AAAAAA== X-Virus-Checked: Checked by ClamAV on apache.org hey all, i was wondering if theres a way to allocate more heap space for each mapper and reducer process that hadoop spawns. i'm getting this error: java.lang.OutOfMemoryError: Java heap space at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:95) at java.io.DataOutputStream.write(DataOutputStream.java:90) at java.io.DataOutputStream.writeUTF(DataOutputStream.java:384) at java.io.DataOutputStream.writeUTF(DataOutputStream.java:306) ... (i've written a custom class to use as the key/value pairs) does anybody know how to allocate more heap space?? thanks!