bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: Bigtop Presentation
Date Thu, 07 Aug 2014 21:17:34 GMT
On Thu, Aug 07, 2014 at 02:55PM, David Fryer wrote:
> Thanks for the feedback!
> Does running puppet apply that way eliminate the need to make a site.csv?

No, site.csv is still required. But it will allow you to set all the nodes
uniformly per the specs from the site.csv. It goes without saying that
site.csv needs to be available to each puppet execution (e.g. on every node).

> What does bigtop_toolchain::packages include?

It sets all needed packages for bigtop development, testing and such. The
package sets are defined within the manifest - just look at it.

Bigtop packages are explicitly declaring their runtime dependency. Which
implies that everything should be already installed by running yum install
(with or without Puppet). Is it your experience that running a Hadoop cluster
requires something different from what bigtop packages are pulling in?
If so - I'd be great to hear of it and fix it if there's a bug somewhere.

> On slide #4 when writing the site.csv, I included the # in there to point
> out that that was a note as opposed to something to be included, although
> that is fairly unclear. What is a better way to make that obvious?

It's more UX point, I guess. People tend to copy-paste code snippets and try
to use them directly. In this case - '#' might be a cause of a later
non-descriptive Puppet error ;) May be just make a comment outside of the code
block?

> Outside of that, I think I've fixed everything else.

Thanks!
  Cos

> On Thu, Aug 7, 2014 at 2:38 PM, Konstantin Boudnik <cos@apache.org> wrote:
> 
> > Thanks for putting this together, David! Let's move into the coming Bigtop
> > book (at some point) ;)
> >
> > Here's a suggestion:
> >  - the slides imply the use of the Bigtop master branch
> >  - which makes the sequence of the steps in slide #3 suboptimal. What I'd
> >    recommend to be done - and it will be less error-prone - is:
> >    1) install puppet
> >    2) clone bigtop and cd into the workspace
> >    3) run
> >         puppet apply --modulepath=. -e "include bigtop_toolchain::packages"
> >
> > Slide #4
> >     step 1)
> >         IIRC '#' in the csv file will break Puppet execution
> >         jdk_package_name,<your java version> - it should be java package
> > name, rather than just a version
> >     step 2) you don't need to make /data be owned by hdfs - only
> > directoies /data/{1..4}
> >     step 3) needs to mention that deployment step has to be run on _each_
> > node.
> >
> > As a practical note, I'd recommend to stay away from using 'jps' command as
> > it's known to have funny issues. 'ps aux | grep java' is usually a
> > bulletproof
> > way to go.
> >
> > Hope it helps,
> >   cos
> >
> > On Thu, Aug 07, 2014 at 09:23AM, David Fryer wrote:
> > > Hi Bigtop!
> > > I think I'm going to do a web presentation on setting up a bare-metal
> > > hadoop cluster. The presentation slides that I will be using can be found
> > > here:
> > >
> > https://docs.google.com/presentation/d/1cEkq2x-Q_n7nbFCttHIn3luoOkz3UMlEEtbudea9l1Y/edit?usp=sharing
> > > If anyone has any edits or comments, I'd be happy to hear your feedback.
> > > Thank You,
> > > David Fryer
> >

Mime
View raw message