Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 221238AC4 for ; Mon, 8 Aug 2011 09:07:46 +0000 (UTC) Received: (qmail 68388 invoked by uid 500); 8 Aug 2011 09:07:45 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 67913 invoked by uid 500); 8 Aug 2011 09:07:35 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 67901 invoked by uid 99); 8 Aug 2011 09:07:31 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 Aug 2011 09:07:31 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of subrotosanyal@huawei.com designates 119.145.14.67 as permitted sender) Received: from [119.145.14.67] (HELO szxga04-in.huawei.com) (119.145.14.67) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 Aug 2011 09:07:25 +0000 Received: from huawei.com (szxga04-in [172.24.2.12]) by szxga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LPL002JNQL86K@szxga04-in.huawei.com> for mapreduce-user@hadoop.apache.org; Mon, 08 Aug 2011 17:05:32 +0800 (CST) Received: from huawei.com ([172.24.2.119]) by szxga04-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTP id <0LPL003XTQL8HD@szxga04-in.huawei.com> for mapreduce-user@hadoop.apache.org; Mon, 08 Aug 2011 17:05:32 +0800 (CST) Received: from blrnshtipl7nc ([10.18.1.37]) by szxml06-in.huawei.com (iPlanet Messaging Server 5.2 HotFix 2.14 (built Aug 8 2006)) with ESMTPA id <0LPL00DDOQL76B@szxml06-in.huawei.com> for mapreduce-user@hadoop.apache.org; Mon, 08 Aug 2011 17:05:32 +0800 (CST) Date: Mon, 08 Aug 2011 14:35:31 +0530 From: Subroto Sanyal Subject: RE: Can I use the cores of each CPU to be the datanodes instead of CPU? In-reply-to: <536e9200.f4d1.131a892335a.Coremail.tanjun_2525@163.com> To: mapreduce-user@hadoop.apache.org Reply-to: subroto.sanyal@huawei.com Message-id: Organization: Htipl MIME-version: 1.0 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.3790.4862 X-Mailer: Microsoft Office Outlook 11 Content-type: multipart/alternative; boundary="Boundary_(ID_ia0DNuoEzqa+RndktwVy4g)" Thread-index: AcxVp/CEpQjFfcQ5SVOpLrotOyEUEgAAcTQg References: <536e9200.f4d1.131a892335a.Coremail.tanjun_2525@163.com> This is a multi-part message in MIME format. --Boundary_(ID_ia0DNuoEzqa+RndktwVy4g) Content-type: text/plain; charset=gb2312 Content-transfer-encoding: quoted-printable =20 Hi Jun, =20 Sorry, I didn=A1=AFt get your question properly=A1=AD.. :-( =20 Are you thinking in terms of Virtual Machines running on same physical server? Are you thinking about CPU core to Hadoop process mapping? =20 As per my understanding, Hadoop doesn=A1=AFt have intelligence of = different cores (ideally application should be independent of hardware details) =20 It is possible to run multiple instance of DataNode in same physical = server, provide the configurations are different for each DataNode process. =20 Regards,=20 Subroto Sanyal _____ =20 From: =CC=B7=BE=FC [mailto:tanjun_2525@163.com]=20 Sent: Monday, August 08, 2011 2:16 PM To: mapreduce Subject: Can I use the cores of each CPU to be the datanodes instead of = CPU? =20 Hi, I have 2 computers each of which is double-core CPU. The 2 computers are datanodes and another computer is the namenode. Now, I want to increase the number of datanode but no more computers available. Can I use the 4 cores of the 2 computers to be 4 datanodes? -- Best regards! Jun Tan --Boundary_(ID_ia0DNuoEzqa+RndktwVy4g) Content-type: text/html; charset=gb2312 Content-transfer-encoding: quoted-printable

 

Hi = Jun,

 

Sorry, I didn=A1=AFt get your = question properly=A1=AD..  L

 

Are you thinking in terms of = Virtual Machines running on same physical server?

Are you thinking about CPU core to = Hadoop process mapping?

 

As per my understanding, Hadoop = doesn=A1=AFt have intelligence of different cores (ideally application should be = independent of hardware details)

 

It is possible to run multiple = instance of DataNode in same physical server, provide the configurations are = different for each DataNode process.

 

= Regards,
Subroto Sanyal

From: = =CC=B7=BE=FC [mailto:tanjun_2525@163.com]
Sent: Monday, August 08, = 2011 2:16 PM
To: mapreduce
Subject: Can I use the = cores of each CPU to be the datanodes instead of = CPU?

 

Hi,

I have 2 computers each of which = is double-core CPU.

The 2 computers are datanodes and = another computer is the namenode.

Now, I want to increase the number = of datanode but no more computers available.

Can I use the 4 cores of the 2 computers to be 4 datanodes?

--

Best = regards!

Jun Tan




--Boundary_(ID_ia0DNuoEzqa+RndktwVy4g)--