hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop Support" <hadoop.supp...@visolve.com>
Subject RE: Multiple separate Hadoop clusters on same physical machines
Date Mon, 02 Feb 2015 11:09:25 GMT
Hello Ashish,

                

Alexander reference is great. Adding to that, you can also find latest Hadoop docker image
from below

 

https://registry.hub.docker.com/u/sequenceiq/hadoop-docker/

 

Set up is as simple as that

1.       Install Docker

2.       Build and Pull above Image

3.       Start Container (if any errors, let us know)

 

Hope it helps!!!

 

Thanks and Regards, 
S.RagavendraGanesh 
Hadoop Support Team 
ViSolve Inc.| <http://www.visolve.com/> www.visolve.com

 

From: Alexander Alten-Lorenz [mailto:wget.null@gmail.com] 
Sent: Monday, February 02, 2015 3:07 PM
To: user@hadoop.apache.org
Cc: Harun Rešit Zafer
Subject: Re: Multiple separate Hadoop clusters on same physical machines

 

http://blog.sequenceiq.com/blog/2014/06/19/multinode-hadoop-cluster-on-docker/

 

Ambari based, but works quite well

 

 

On 02 Feb 2015, at 10:33, Ashish Kumar9 <ashishk4@in.ibm.com <mailto:ashishk4@in.ibm.com>
> wrote:

 

Is there any good reference material available to follow to test docker and hadoop integration
. 




From:        <hadoop.support@visolve.com <mailto:hadoop.support@visolve.com> >

To:        "'Harun Reşit Zafer'" <harun.zafer@tubitak.gov.tr <mailto:harun.zafer@tubitak.gov.tr>
>, <user@hadoop.apache.org <mailto:user@hadoop.apache.org> > 
Date:        02/02/2015 02:57 PM 
Subject:        RE: Multiple separate Hadoop clusters on same physical machines 

  _____  




Hello Harun, 
  
Your question is very interesting and will be useful for future Hadoop setups for startup/individuals
too. 
  
Normally for testing purposes, we prefer you to use pseudo-distributed environments (i.e.
installation of all cluster files in single node). You can refer few links which will guide
you through the whole process below for reference: 
  
 <https://districtdatalabs.silvrback.com/creating-a-hadoop-pseudo-distributed-environment>
https://districtdatalabs.silvrback.com/creating-a-hadoop-pseudo-distributed-environment 
  
Individual Pseudo Distributed Cluster Implementation: 
  
 <http://www.thegeekstuff.com/2012/02/hadoop-pseudo-distributed-installation/> http://www.thegeekstuff.com/2012/02/hadoop-pseudo-distributed-installation/

 <http://hbase.apache.org/book.html#quickstart_pseudo> http://hbase.apache.org/book.html#quickstart_pseudo

and please check for others. 
  
>From our 20 years of Server & its related Industrial experience, we recommend you
to use VM/Instances for production & Business Critical environment. Other way around,
if you are developing some products related to Hadoop, you can use docker & other related
resources for development. As shipment to production will become stress free with the use
of these tools with cluster environment setup. 
  
Feel free to ask for further queries. 
  
Thanks and Regards, 
S.RagavendraGanesh 
Hadoop Support Team 
ViSolve Inc.| <http://www.visolve.com/> www.visolve.com 
  
  
  
From: Alexander Pivovarov [ <mailto:apivovarov@gmail.com> mailto:apivovarov@gmail.com]

Sent: Monday, February 02, 2015 12:56 PM
To: user@hadoop.apache.org <mailto:user@hadoop.apache.org> 
Subject: Re: Multiple separate Hadoop clusters on same physical machines 
  
start several vms and install hadoop on each vm 
keywords: kvm, QEMU 
  
On Mon, Jan 26, 2015 at 1:18 AM, Harun Reşit Zafer <harun.zafer@tubitak.gov.tr <mailto:harun.zafer@tubitak.gov.tr>
> wrote: 
Hi everyone,

We have set up and been playing with Hadoop 1.2.x and its friends (Hbase, pig, hive etc.)
on 7 physical servers. We want to test Hadoop (maybe different versions) and ecosystem on
physical machines (virtualization is not an option) from different perspectives.

As a bunch of developer we would like to work in parallel. We want every team member play
with his/her own cluster. However we have limited amount of servers (strong machines though).

So the question is, by changing port numbers, environment variables and other configuration
parameters, is it possible to setup several independent clusters on same physical machines.
Is there any constraints? What are the possible difficulties we are to face?

Thanks in advance

-- 
Harun Reşit Zafer
TÜBİTAK BİLGEM BTE
Bulut Bilişim ve Büyük Veri Analiz Sistemleri Bölümü
T +90 262 675 3268 <tel:%2B90%20262%20675%203268> 
W  http://www.hrzafer.com <http://www.hrzafer.com/>  
  

 


Mime
View raw message