Return-Path: Delivered-To: apmail-hadoop-core-user-archive@www.apache.org Received: (qmail 47439 invoked from network); 1 Jul 2008 02:53:02 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 1 Jul 2008 02:53:02 -0000 Received: (qmail 11560 invoked by uid 500); 1 Jul 2008 02:52:59 -0000 Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org Received: (qmail 11526 invoked by uid 500); 1 Jul 2008 02:52:59 -0000 Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-user@hadoop.apache.org Delivered-To: mailing list core-user@hadoop.apache.org Received: (qmail 11514 invoked by uid 99); 1 Jul 2008 02:52:59 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Jun 2008 19:52:59 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of wangchihhs@gmail.com designates 209.85.198.235 as permitted sender) Received: from [209.85.198.235] (HELO rv-out-0506.google.com) (209.85.198.235) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Jul 2008 02:52:09 +0000 Received: by rv-out-0506.google.com with SMTP id k40so2986950rvb.29 for ; Mon, 30 Jun 2008 19:52:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type; bh=0NsAc54cmY2tROeLQ6hEFKIZN9k6pTurzjfV1Di6Mf8=; b=EkutBkJKxrPxnNOVXEpZAf/J84vZc9qLh3FCBgnGTm0zqndI/IlfoDmQkCQHZ1Hi7r G7v7iC0FP0hqBFLAYGo/oSc2wTYUEdKW0AAbM7aznzW+kD9aBx0BQ0UEVcAtVd/F5omL SxsTMRwMgxeHs0oPXbiNCIwHRbZZ7MZiWeICM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=WibNz3inW7a4SIM9/tjFnuQzmxbkfBnOTYpGAaQ+Fte+zszX6C2gGgwJJpmcfbQPfL pG+Cm+UfCddqLE/BP5YCbVykwE/zgC6opSauavj5ohnmZbs/EwgRJiGBlhQKhAUW89Z3 ZTBUy58scA1Z78zA1cbFb9aWxojL89/lMewnY= Received: by 10.141.79.12 with SMTP id g12mr3162389rvl.29.1214880749504; Mon, 30 Jun 2008 19:52:29 -0700 (PDT) Received: by 10.141.211.8 with HTTP; Mon, 30 Jun 2008 19:52:29 -0700 (PDT) Message-ID: <86ada3830806301952h74871e26x643faead5315c3a3@mail.gmail.com> Date: Tue, 1 Jul 2008 10:52:29 +0800 From: "=?BIG5?B?pP2n07K7?=" To: core-user@hadoop.apache.org Subject: Test Hadoop performance on EC2 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_16021_13309749.1214880749486" X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_16021_13309749.1214880749486 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi all, I've made some experiments on Hadoop on Amazon EC2. I would like to share the result and any feedback would be appreciated. Environment: -Xen VM (Amazon EC2 instance ami-ee53b687) -1.7Ghz Xeon CPU, 1.75GB of RAM, 160GB of local disk, and 250Mb/s of network bandwidth (small instance) -Hadoop 0.17.0 -storage: HDFS -Test example: wordcount Experiment 1: (fixed # of instances (8), variant data size (2MB~512MB), # of maps: 8, # of reduces: 8) Data Size(MB) | Time(s) 128 256 512 Experiment 2: Experiment 3: -- Best Regards, Shawn ------=_Part_16021_13309749.1214880749486--