bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Demai Ni <>
Subject Re: newbie question of BigTop
Date Tue, 09 Jun 2015 00:02:14 GMT

thank you very much for the detail instructions.

I understood that the cluster need to be passwordless ssh or someway to
provide userid/password of all nodes. So that actions can be propogated to
other nodes.

I was expecting some tool (pdsh, vagrant provisioner) as you mentioned,
would do such job for end-users. But the document didn't mention such
tool(or I missed them?), and I got the impression that that steps 1)~3) are
required by manual effort.

Maybe those tools are common-sense for an experienced cluster administer,
so no document is necessary.:-) Anyway, now that with your help, I will try
them out. Thanks


On Sat, Jun 6, 2015 at 12:03 PM, Evans Ye <> wrote:

> The missing note in the document is that the cluster admin need to setup
> some general environment settings such as
> the password-less ssh. Definitely it would be great if we can provide some
> suggestions or ways to do that, but I think somehow you still need to setup
> that during machine provisioning(installing OS, setting up hostname,
> ip, password, ssh keys), right?
> If you have bunch of machines ready for the deployment, here's the step
> to setup a bigtop cluster:
> 1) install puppet on each node
> 2) install puppet modules on each node
> puppet apply --modulepath= -e "include bigtop_toolchain::puppet-modules"
> 2) propagate puppet recipes and configurations
> <>on
> each node
> 3) run puppet apply on each node
> 4) done!
> The puppet and module installation can be done through cluster admin tool
> such as pdsh, which allows you to run same command across the whole
> cluster in parallel mode. The puppet recipes or bigtop repo can be put in
> a share storage such as NFS so that it is available for all nodes. After
> all of this are set, you can then run puppet apply on each node through
> pdsh as well. This should give you an up and running bigtop cluster.
> If you're not quite sure how to do that, you can also refer to bigtop's vagrant
> provisioner
> <>,
> which demonstrate the whole process to setup a bigtop cluster by a click of
> button.
> Of course, you can ask questions in mailing list when you encounter any
> issue. We're happy to help. :)
> 2015-06-06 6:30 GMT+08:00 Demai Ni <>:
>> hi, folks,
>> I am looking at BigTop as noncommercial hadoop distribution package, as
>> an alternative of CDH, HDP. And this is my first time to try out bigTop.
>> My expectation was:
>> 1) setup my cluster with passwordless ssh
>> 2) download the necessary bigtop components on one of the nodes.
>> 3) run an installation/configure tool from the node on 2)
>> 4) BigTop will (through puppet) install and configure the whole cluster.
>> the point is that no need of a lot of manual work on every node.
>> But I am following instruction for 0.8.0 and 0.7.0, like
>> there are quite a few steps, (almost all the steps in above link) are
>> required to be performed on each node?
>> so it looks like the human work is significant higher between small(3
>> node) and large(100+) cluster.  Is that right?
>> Thanks.
>> Demai

View raw message