bigtop-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From evan...@apache.org
Subject bigtop git commit: BIGTOP-1914. improve puppet README.md file
Date Sat, 04 Jul 2015 18:50:20 GMT
Repository: bigtop
Updated Branches:
  refs/heads/master 362ca725e -> 6fd647e9c


BIGTOP-1914. improve puppet README.md file

Signed-off-by: Evans Ye <evansye@apache.org>


Project: http://git-wip-us.apache.org/repos/asf/bigtop/repo
Commit: http://git-wip-us.apache.org/repos/asf/bigtop/commit/6fd647e9
Tree: http://git-wip-us.apache.org/repos/asf/bigtop/tree/6fd647e9
Diff: http://git-wip-us.apache.org/repos/asf/bigtop/diff/6fd647e9

Branch: refs/heads/master
Commit: 6fd647e9c5a5f7929f36482dbf780ea2ce61bae6
Parents: 362ca72
Author: Martin Bukatovic <mbukatov@redhat.com>
Authored: Thu Jul 2 13:41:55 2015 +0200
Committer: Evans Ye <evansye@apache.org>
Committed: Sat Jul 4 18:49:41 2015 +0000

----------------------------------------------------------------------
 bigtop-deploy/puppet/README.md | 112 +++++++++++++++++-------------------
 1 file changed, 52 insertions(+), 60 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/bigtop/blob/6fd647e9/bigtop-deploy/puppet/README.md
----------------------------------------------------------------------
diff --git a/bigtop-deploy/puppet/README.md b/bigtop-deploy/puppet/README.md
index 634a820..1e9c68a 100644
--- a/bigtop-deploy/puppet/README.md
+++ b/bigtop-deploy/puppet/README.md
@@ -5,102 +5,89 @@
 BigTop is now using Puppet 3.x (BIGTOP-1047)!
 
 Installing and starting hadoop services is non-trivial, and for this reason bigtop 
-
 packages puppet instrumentation for the various ecosystem components, which works
-
 synergistically with bigtop produced software packages.
 
 The puppet classes for bigtop deployment setup and deploy hadoop services.
-
 This includes tasks such as:
 
 - service installation
 - pointing slaves to masters (i.e. regionservers, nodemanagers to their respective master)
 - starting the services
 
-The mode of puppet is masterless : there is no fancy coordination happening behind the scenes.
+The mode of puppet is *masterless*: there is no fancy coordination happening behind the scenes.
 
 Puppet has a notion of a configuration directory, called config.  
+When running puppet apply, note that puppet's confdir is *underneath* the `--confdir` value.
+For example: If you have `site.csv` in `/etc/puppet/config`, 
+Then you should use `--confdir=/etc/puppet` , and puppet finds the config dir underneath.
 
-When running puppet apply, note that puppet's confdir is *underneath* the --confdir value.
-
-For example: 
-
-If you have site.csv in /etc/puppet/config, 
-
-Then you should use --confdir=/etc/puppet , and puppet finds the config dir underneath.
-
-As an end to end example, you can follow the vagrant-puppet-vm recipes to see how to set
up
-
+As an end to end example, you can follow the `vagrant-puppet-vm` recipes to see how to set
up
 a puppet managed bigtop hadoop installation.  Those examples are gauranteed to work and 
-
 serve as a pedagogical round trip to the way bigtop integrates packaging, deployment, and

-
 testing all into one package.
 
 ## Debugging
 
-If in any case, you need to debug these recipes, you can add notify("...") statements into

+If in any case, you need to debug these recipes, you can add `notify("...")` statements into

 the puppet scripts.  
 
 In time, we will add more logging and debugging to these recipes.  Feel free to submit 
-
-a patch for this !
+a patch for this!
 
 ## Configuration
 
-As above, we defined a confdir (i.e. /etc/puppet/) which has a config/ directory in it.
-
-The heart of puppet is the manifests file.  This file ( manifests/init.pp ) 
+As above, we defined a confdir (i.e. `/etc/puppet/`) which has a `config/` directory in it.
 
-expects configuration to live in hiera as specified by $confdir/hiera.yaml. An example
-hiera.yaml as well as hiera configuration yaml files are provided with the bigtop classes.
They
+The heart of puppet is the manifests file.  This file (`manifests/init.pp`) 
+expects configuration to live in hiera as specified by `$confdir/hiera.yaml`. An example
+`hiera.yaml` as well as hiera configuration yaml files are provided with the bigtop classes.
They
 basically take the form:
 
-<pre>
+```
 key: value
-</pre>
+```
 
 with syntactic variations for hashes and arrays. Please consult the excellent puppet and
hiera
 documentation for details.
 
-All configuration is done via such key value assignments in hierdata/site.yaml.  Any options
-not defined there will revert to a default value defined in hieradata/cluster.yaml, with
the
+All configuration is done via such key value assignments in `hierdata/site.yaml`. Any options
+not defined there will revert to a default value defined in `hieradata/cluster.yaml`, with
the
 following exceptions (which are required):
 
-* bigtop::hadoop\_head\_node: must be set to the FQDN of the name node of your
-	cluster (which will also become its job tracker and gateway)
+* `bigtop::hadoop_head_node`: must be set to the FQDN of the name node of your
+   cluster (which will also become its job tracker and gateway)
 
-* bigtop::bigtop\_repo\_uri: uri of a repository containing packages for
-	hadoop as built by Bigtop.
+* `bigtop::bigtop_repo_uri`: uri of a repository containing packages for
+   hadoop as built by Bigtop.
 
-$confdir is the directory that puppet will look into for its configuration.  On most systems,

-this will be either /etc/puppet/ or /etc/puppetlabs/puppet/.  You may override this value
by 
-specifying --confdir=path/to/config/dir on the puppet command line.
+`$confdir` is the directory that puppet will look into for its configuration.  On most systems,

+this will be either `/etc/puppet/` or `/etc/puppetlabs/puppet/`.  You may override this value
by 
+specifying `--confdir=path/to/config/dir` on the puppet command line.
 
-cluster.yaml also serves as an example what parameters can be set and how they usually interact
+`cluster.yaml` also serves as an example what parameters can be set and how they usually
interact
 between modules.
 
 You can instruct the recipes to install ssh-keys for user hdfs to enable passwordless login
 across the cluster. This is for test purposes only, so by default the option is turned off.
 
-Files such as ssh-keys are imported from the master using the puppet:/// URL scheme. For
this
+Files such as ssh-keys are imported from the master using the `puppet:///` URL scheme. For
this
 to work, fileserver has to be enabled on the puppet master, the files module enabled and
access
 allowed in auth.conf. fileserver.conf should therefore contain e.g.:
 
-<pre>
+```
 [files]
   path /etc/puppet/files
   allow *
-</pre>
+```
 
 No changes are required to the default puppet 3 auth.conf.
 
 For other options that may be set here, look for class parameters in the modules'
 manifests/init.pp files. Any class parameter can be used as a hiera key if prefixed with
the
 module and class namespace. Module hue's server class will look for its parameter rm_host
as
-hue::server::rm_host in hiera.
-Note that if hadoop::hadoop\_storage\_dirs is left unset, puppet will attempt to guess which
+`hue::server::rm_host` in hiera.
+Note that if `hadoop::hadoop_storage_dirs` is left unset, puppet will attempt to guess which
 directories to use.
 
 ## Usage
@@ -109,13 +96,18 @@ directories to use.
 - Make sure puppet is installed
 - Make sure all the required puppet modules are installed:
 
-<pre>
-gradle toolchain-puppetmodules (if you already have JAVA installed)
+```
+gradle toolchain-puppetmodules # if you already have JAVA installed
+```
+
 or
+
+```
 puppet apply --modulepath=<path_to_bigtop> -e "include bigtop_toolchain::puppet-modules"
-</pre>
+```
 
 This will install the following module(s) for you:
+
   * [puppet stdlib module](https://forge.puppetlabs.com/puppetlabs/stdlib)
   * [puppet apt module](https://forge.puppetlabs.com/puppetlabs/apt) on Ubuntu, Debian only
 
@@ -124,11 +116,11 @@ Bigtop toolchan can take care of that for you, so just be aware of it.
 
 And run the following on those nodes:
 
-<pre>
+```
 # cp bigtop-deploy/puppet/hiera.yaml /etc/puppet
 # mkdir -p /etc/puppet/hieradata
 # rsync -a --delete bigtop-deploy/puppet/hieradata/bigtop/ /etc/puppet/hieradata/bigtop/
-# cat > /etc/puppet/hieradata/site.yaml &lt;&lt; EOF
+# cat > /etc/puppet/hieradata/site.yaml <<EOF
 # bigtop::hadoop_head_node: "hadoopmaster.example.com"
 # hadoop::hadoop_storage_dirs:
 #   - "/data/1"
@@ -136,13 +128,13 @@ And run the following on those nodes:
 # bigtop::bigtop_repo_uri: "http://mirror.example.com/path/to/mirror/"
 # EOF
 # puppet apply -d --modulepath="bigtop-deploy/puppet/modules:/etc/puppet/modules" bigtop-deploy/puppet/manifests/site.pp
-</pre>
+```
 
 When ignite-hadoop accelerator is deployed the client configs are placed under
-/etc/hadoop/ignite.client.conf. All one needs to do to run Mapreduce jobs on ignite-hadoop
grid
-is to set HADOOP_CONF_DIR=/etc/hadoop/ignite.client.conf in the client session.
+`/etc/hadoop/ignite.client.conf`. All one needs to do to run Mapreduce jobs on ignite-hadoop
grid
+is to set `HADOOP_CONF_DIR=/etc/hadoop/ignite.client.conf` in the client session.
 
-# Passwords
+## Passwords
 
 These classes are mostly used for regression testing. For ease of use they
 contain insecure default passwords in a number of places. If you intend to use
@@ -150,10 +142,10 @@ them in production environments, make sure to track down all those places
and
 set proper passwords. This can be done using the corresponding hiera settings.
 Some of these (but almost certainly not all!) are:
 
-<pre>
+```
 hadoop::common_hdfs::hadoop_http_authentication_signature_secret
 hadoop::httpfs::secret
-</pre>
+```
 
 ## Automatic password generation
 
@@ -175,11 +167,11 @@ with this approach at the moment:
 
 trocla needs to be installed on the master only. To do so, run the following:
 
-<pre>
+```
 # gem install trocla
 # puppet module install duritong/trocla
 # puppet apply -e "class { 'trocla::config': manage_dependencies => false }"
-</pre>
+```
 
 The trocla ruby gem pulls in highline, moneta and bcrypt. The bcrypt gem needs
 ruby development packages (ruby.h) and a compiler.
@@ -187,9 +179,9 @@ ruby development packages (ruby.h) and a compiler.
 Alternatively you can install your distributions' binary packages *before*
 running gem install. On Debian those packages can be installed as follows:
 
-<pre>
+```
 apt-get install ruby-highline ruby-moneta ruby-bcrypt
-</pre>
+```
 
 This installation process is expected to get easier once operation system
 packages have been created. This is actively underway for Debian. See Debian
@@ -198,14 +190,14 @@ bugs #777761 and #777906 for progress.
 After installing the trocla gem as outlined above, the following test should
 work:
 
-<pre>
+```
 # puppet apply -e "file { '/tmp/test': content => trocla("test", "plain") }"
 # cat /tmp/test
 puGNOX-G%zYDKHet
-</pre>
+```
 
 Now, automatic password generation can be activated in site.yaml using
 
-<pre>
+```
 hadoop::generate_secrets: true
-</pre>
+```


Mime
View raw message