hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bruno Mahé <bm...@apache.org>
Subject Re: Which hadoop installation should I use on ubuntu server?
Date Fri, 29 Mar 2013 09:59:27 GMT
On 03/29/2013 01:09 AM, David Parks wrote:
> Hmm, seems intriguing. I’m still not totally clear on bigtop here. It
> seems like they’re creating and maintain basically an installer for Hadoop?
>
> I tried following their docs for Ubuntu, but just get a 404 error on the
> first step, so it makes me wonder how reliable that project is.
>
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop
>
> Has anyone actually used bigtop to deploy Hadoop in a production
> environment?
>


Hi David,

You may want to send an email to Apache Bigtop mailing lists with your 
questions, but the goal of Apache Bigtop is three folds:
1/ Provide top notch packages for Apache Hadoop related projects for the 
most popular GNU/Linux distributions
2/ Provide a point of integration and testing for all these projects
3/ Provide means to reliably deploy a complete stack.

Apache Bigtop was donated by Cloudera to the Apache Foundation and is
now used as the base for CDH (Cloudera's distribution).

All the projects supported by Apache Bigtop have packages for 
Debian/Ubuntu/SLES11/Fedora/CentOS.
They also have tests to exercise integration points between all of them
(ex: Hive can use HBase which sits on top of HDFS). And in order to run
these tests, we also have a test framework.
Also before we can test for integration, we also have to ensure they can 
be properly installed/upgraded/removed, with the right users, ulimits, 
rights and so forth. So to that end, we also have a large chunk of the 
tests and testing framework dedicated to testing the packages themselves.

And finally, regarding the deployment, we have the following:
* Boxgrinder recipe people can use and modify to suit their need. This 
is useful if you want to create your own virtual machine (kvm, vmware, 
virtualbox) but also want to create images for ec2.
* kickstart file to build a live fedora image
* puppet recipes to deploy all these services.


Regarding your issue with the instructions on the wiki, this is because 
they have not been updated since Apache Bigtop became a top level 
project, and the location of the artefacts has changed. When reading the 
instructions, please use the following as the base url: 
http://www.apache.org/dist/bigtop/bigtop-0.5.0/

Please, let me know if you still have some issues.

Thanks,
Bruno

Mime
View raw message