hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish Kumar9 <ashis...@in.ibm.com>
Subject RE: Multiple separate Hadoop clusters on same physical machines
Date Mon, 02 Feb 2015 09:33:04 GMT
Is there any good reference material available to follow to test docker 
and hadoop integration .

From:   <hadoop.support@visolve.com>
To:     "'Harun Reşit Zafer'" <harun.zafer@tubitak.gov.tr>, 
Date:   02/02/2015 02:57 PM
Subject:        RE: Multiple separate Hadoop clusters on same physical 

Hello Harun,
Your question is very interesting and will be useful for future Hadoop 
setups for startup/individuals too.
Normally for testing purposes, we prefer you to use pseudo-distributed 
environments (i.e. installation of all cluster files in single node). You 
can refer few links which will guide you through the whole process below 
for reference:
Individual Pseudo Distributed Cluster Implementation:
and please check for others.
>From our 20 years of Server & its related Industrial experience, we 
recommend you to use VM/Instances for production & Business Critical 
environment. Other way around, if you are developing some products related 
to Hadoop, you can use docker & other related resources for development. 
As shipment to production will become stress free with the use of these 
tools with cluster environment setup. 
Feel free to ask for further queries.
Thanks and Regards,
Hadoop Support Team
ViSolve Inc.|www.visolve.com
From: Alexander Pivovarov [mailto:apivovarov@gmail.com] 
Sent: Monday, February 02, 2015 12:56 PM
To: user@hadoop.apache.org
Subject: Re: Multiple separate Hadoop clusters on same physical machines
start several vms and install hadoop on each vm
keywords: kvm, QEMU
On Mon, Jan 26, 2015 at 1:18 AM, Harun Reşit Zafer <
harun.zafer@tubitak.gov.tr> wrote:
Hi everyone,

We have set up and been playing with Hadoop 1.2.x and its friends (Hbase, 
pig, hive etc.) on 7 physical servers. We want to test Hadoop (maybe 
different versions) and ecosystem on physical machines (virtualization is 
not an option) from different perspectives.

As a bunch of developer we would like to work in parallel. We want every 
team member play with his/her own cluster. However we have limited amount 
of servers (strong machines though).

So the question is, by changing port numbers, environment variables and 
other configuration parameters, is it possible to setup several 
independent clusters on same physical machines. Is there any constraints? 
What are the possible difficulties we are to face?

Thanks in advance

Harun Reşit Zafer
Bulut Bilişim ve Büyük Veri Analiz Sistemleri Bölümü
T +90 262 675 3268
W  http://www.hrzafer.com

View raw message