bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Demai Ni <nid...@gmail.com>
Subject Re: newbie question of BigTop
Date Tue, 09 Jun 2015 00:02:14 GMT
Evans,

thank you very much for the detail instructions.

I understood that the cluster need to be passwordless ssh or someway to
provide userid/password of all nodes. So that actions can be propogated to
other nodes.

I was expecting some tool (pdsh, vagrant provisioner) as you mentioned,
would do such job for end-users. But the document didn't mention such
tool(or I missed them?), and I got the impression that that steps 1)~3) are
required by manual effort.

Maybe those tools are common-sense for an experienced cluster administer,
so no document is necessary.:-) Anyway, now that with your help, I will try
them out. Thanks

Demai

On Sat, Jun 6, 2015 at 12:03 PM, Evans Ye <evansye@apache.org> wrote:

> The missing note in the document is that the cluster admin need to setup
> some general environment settings such as
> the password-less ssh. Definitely it would be great if we can provide some
> suggestions or ways to do that, but I think somehow you still need to setup
> that during machine provisioning(installing OS, setting up hostname,
> ip, password, ssh keys), right?
>
> If you have bunch of machines ready for the deployment, here's the step
> to setup a bigtop cluster:
>
> 1) install puppet on each node
> 2) install puppet modules on each node
>
> puppet apply --modulepath= -e "include bigtop_toolchain::puppet-modules"
>
> 2) propagate puppet recipes and configurations
> <https://github.com/apache/bigtop/tree/master/bigtop-deploy/puppet>on
> each node
> 3) run puppet apply on each node
> 4) done!
>
> The puppet and module installation can be done through cluster admin tool
> such as pdsh, which allows you to run same command across the whole
> cluster in parallel mode. The puppet recipes or bigtop repo can be put in
> a share storage such as NFS so that it is available for all nodes. After
> all of this are set, you can then run puppet apply on each node through
> pdsh as well. This should give you an up and running bigtop cluster.
>
> If you're not quite sure how to do that, you can also refer to bigtop's vagrant
> provisioner
> <https://github.com/apache/bigtop/tree/master/bigtop-deploy/vm/vagrant-puppet-vm>,
> which demonstrate the whole process to setup a bigtop cluster by a click of
> button.
>
> Of course, you can ask questions in mailing list when you encounter any
> issue. We're happy to help. :)
>
> 2015-06-06 6:30 GMT+08:00 Demai Ni <nidmgg@gmail.com>:
>
>> hi, folks,
>>
>> I am looking at BigTop as noncommercial hadoop distribution package, as
>> an alternative of CDH, HDP. And this is my first time to try out bigTop.
>>
>> My expectation was:
>> 1) setup my cluster with passwordless ssh
>> 2) download the necessary bigtop components on one of the nodes.
>> 3) run an installation/configure tool from the node on 2)
>> 4) BigTop will (through puppet) install and configure the whole cluster.
>>
>> the point is that no need of a lot of manual work on every node.
>>
>> But I am following instruction for 0.8.0 and 0.7.0, like
>>
>> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+BigTop+0.7.0+hadoop+on+CentOS+with+puppet
>>
>> there are quite a few steps, (almost all the steps in above link) are
>> required to be performed on each node?
>>
>> so it looks like the human work is significant higher between small(3
>> node) and large(100+) cluster.  Is that right?
>>
>> Thanks.
>>
>> Demai
>>
>>
>
>

Mime
View raw message