bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Grover <m...@apache.org>
Subject Re: New to Bigtop, where to start?
Date Wed, 16 Jul 2014 15:34:03 GMT
Welcome, David.

For physical machines, I personally always use instructions like these:
https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop+0.6.0

These for Bigtop 0.6.0, the latest Bigtop release is 0.7.0 but we don't
have a page for that unfortunately (we should and if you could help with
that, that'd be much appreciated!). We are tying up lose ends for Bigtop
0.8, so we hope to release it soon.

Mark


On Wed, Jul 16, 2014 at 8:20 AM, jay vyas <jayunit100.apache@gmail.com>
wrote:

> one more note : by "look at the csv file" above i meant, "edit it so that
> it reflects your
> environment".
>
> Make sure and read  the puppet README file as well under
> bigtop-deploy/puppet.
>
>
> On Wed, Jul 16, 2014 at 11:15 AM, jay vyas <jayunit100.apache@gmail.com>
> wrote:
>
>> Hi david .
>>
>> Glad to hear the vagrant stuff worked for you.  Now , the next step will
>> be to port it to bare metal, like you say.
>>
>> The Vagrantfile does two things
>>
>> 1) It creates a shared folder for all machines.
>> 2) It spins up centos boxes .
>>
>>
>> So in the "real world" you will need to obviously set up ssh between
>> machines to start.
>> After that , roughly, will need to do the following:
>>
>> - clone bigtop onto each of your  machines
>> - install puppet 2.x on each of the machines
>> - look at the csv file created in the vagrant provisioner, and read the
>> puppet README file (in bigtop-deploy)
>> - run puppet apply on the head node
>> Once that works
>> - run puppet apply on each slave.
>> now on any node that you use as client, (i just use the master usually)
>> you can yum install your favorite ecosystem components:
>> yum install -y pig mahout
>>
>> And you have a working hadoop cluster.
>>
>> one idea as I know your on the east coast, if your company is interested
>> in hosting/sponsoring a bigtop meetup, we could possibly bring some folks
>> from the boston / nyc area together to walk through building a bigtop
>> cluster on bare metal.  Let us know if any other questions.   These
>> directions are admittedly a little bit rough.
>>
>> Also, once you get this working, you can help us to update the wiki pages.
>>
>>
>>
>>
>>
>>
>> On Wed, Jul 16, 2014 at 10:39 AM, David Fryer <dfryer1193@gmail.com>
>> wrote:
>>
>>> Hi Bigtop!
>>>
>>> I'm looking to use bigtop to help set up a small hadoop cluster. I'm
>>> currently messing about with the hadoop tarball and all of the associated
>>> xml files, and I don't really have the time or expertise to get it up and
>>> working.
>>>
>>> Jay suggested that bigtop may be a good solution, so I've decided to
>>> give it a shot. Unfortunately, documentation is fairly sparse and I'm not
>>> quite sure where to start. I've cloned the github repo and used the
>>> startup.sh script found in bigtop/bigtop-deploy/vm/vagrant-puppet to set up
>>> a virtual cluster, but I am unsure how to apply this to physical machines.
>>> I'm also not quite sure how to get hadoop and hdfs up and working.
>>>
>>> Any help would be appreciated!
>>>
>>> Thanks,
>>> David Fryer
>>>
>>
>>
>>
>> --
>> jay vyas
>>
>
>
>
> --
> jay vyas
>

Mime
View raw message