From hadoop-dev-return-9088-apmail-lucene-hadoop-dev-archive=lucene.apache.org@lucene.apache.org Mon Mar 19 02:13:52 2007 Return-Path: Delivered-To: apmail-lucene-hadoop-dev-archive@locus.apache.org Received: (qmail 14379 invoked from network); 19 Mar 2007 02:13:52 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 19 Mar 2007 02:13:52 -0000 Received: (qmail 25758 invoked by uid 500); 19 Mar 2007 02:13:59 -0000 Delivered-To: apmail-lucene-hadoop-dev-archive@lucene.apache.org Received: (qmail 25725 invoked by uid 500); 19 Mar 2007 02:13:59 -0000 Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-dev@lucene.apache.org Received: (qmail 25716 invoked by uid 99); 19 Mar 2007 02:13:58 -0000 Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 18 Mar 2007 19:13:58 -0700 X-ASF-Spam-Status: No, hits=2.0 required=10.0 tests=HTML_MESSAGE,SPF_HELO_PASS X-Spam-Check-By: apache.org Received-SPF: unknown ~allip4:220.95.234.208 (herse.apache.org: encountered unrecognized mechanism during SPF processing of domain of 1004shi@nhncorp.com) Received: from [121.156.78.140] (HELO mexkrf02.all.nhncorp.com) (121.156.78.140) by apache.org (qpsmtpd/0.29) with SMTP; Sun, 18 Mar 2007 19:13:49 -0700 Received: from MEXKR02.all.nhncorp.com ([10.30.3.13]) by mexkrf02.all.nhncorp.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 19 Mar 2007 11:13:27 +0900 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C769CC.2E15E89E" Subject: Initial number of Maps on the machine Date: Mon, 19 Mar 2007 11:13:27 +0900 Message-ID: <659480E7BA17624988C9484785AC1F6B028BA6@MEXKR02.all.nhncorp.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Initial number of Maps on the machine Thread-Index: Acdm79XvoT9wX1RETOSD8bDMS29U3gC1qz9T References: <45E69FA9.9010604@lonecrusader.co.uk> <45E6DC68.4030200@lonecrusader.co.uk> <45E7E56F.5060208@lonecrusader.co.uk> <45E82164.80400@pagesmiths.com> <2E7EB709-BD97-4F78-BABC-E879D7E65E80@toolazydogs.com> <45F924F9.6090400@lonecrusader.co.uk> From: =?ks_c_5601-1987?B?vcnFubHm?= <1004shi@nhncorp.com> To: X-OriginalArrivalTime: 19 Mar 2007 02:13:27.0904 (UTC) FILETIME=[2E93EE00:01C769CC] X-Virus-Checked: Checked by ClamAV on apache.org ------_=_NextPart_001_01C769CC.2E15E89E Content-Type: text/plain; charset="ks_c_5601-1987" Content-Transfer-Encoding: quoted-printable When I run Simple MR Job such as grep (about 200Maps & 4 Reduces) with = 20 Opeteron Servers ( 2Way Dual-Core, 4GB RAM) Only 2 maps are instatantiated on one machine and each map task takes = 5~6 seconds to be done. =20 As a result, about 50% CPU is unsed during the MR Job and the overall = performance is not that good as I expected=20 Configuration "mapred.tasktracker.tasks.maximum =3D=3D> 10", It only = works when each map tasks last more than 10 seconds It seemds that Hadoop framework starts MR job with the limit of 2 maps = on one machine. =20 When running two similar MR Jobs concurrently, the number of maps on one = machine is still 2 and CPU usage is about 50% and each MR Jobs takes = almost 2times longer to be done. Then, How can I change the initial limit of map's count on the machine ? =20 ------_=_NextPart_001_01C769CC.2E15E89E--