bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jay vyas <>
Subject Re: New to Bigtop, where to start?
Date Wed, 16 Jul 2014 15:15:54 GMT
Hi david .

Glad to hear the vagrant stuff worked for you.  Now , the next step will be
to port it to bare metal, like you say.

The Vagrantfile does two things

1) It creates a shared folder for all machines.
2) It spins up centos boxes .

So in the "real world" you will need to obviously set up ssh between
machines to start.
After that , roughly, will need to do the following:

- clone bigtop onto each of your  machines
- install puppet 2.x on each of the machines
- look at the csv file created in the vagrant provisioner, and read the
puppet README file (in bigtop-deploy)
- run puppet apply on the head node
Once that works
- run puppet apply on each slave.
now on any node that you use as client, (i just use the master usually) you
can yum install your favorite ecosystem components:
yum install -y pig mahout

And you have a working hadoop cluster.

one idea as I know your on the east coast, if your company is interested in
hosting/sponsoring a bigtop meetup, we could possibly bring some folks from
the boston / nyc area together to walk through building a bigtop cluster on
bare metal.  Let us know if any other questions.   These directions are
admittedly a little bit rough.

Also, once you get this working, you can help us to update the wiki pages.

On Wed, Jul 16, 2014 at 10:39 AM, David Fryer <> wrote:

> Hi Bigtop!
> I'm looking to use bigtop to help set up a small hadoop cluster. I'm
> currently messing about with the hadoop tarball and all of the associated
> xml files, and I don't really have the time or expertise to get it up and
> working.
> Jay suggested that bigtop may be a good solution, so I've decided to give
> it a shot. Unfortunately, documentation is fairly sparse and I'm not quite
> sure where to start. I've cloned the github repo and used the
> script found in bigtop/bigtop-deploy/vm/vagrant-puppet to set up a virtual
> cluster, but I am unsure how to apply this to physical machines. I'm also
> not quite sure how to get hadoop and hdfs up and working.
> Any help would be appreciated!
> Thanks,
> David Fryer

jay vyas

View raw message