hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jay vyas <jayunit100.apa...@gmail.com>
Subject Re: Hadoop virtual machine
Date Sun, 06 Jul 2014 18:01:24 GMT
I really like the Cascading recipes above, thanks for sharing that !

Also we have *apache bigtop vagrant recipes* which we curate for this kind
of thing, which are really useful.  You can spin up a 1 or multi node
cluster, just by running the startup.sh script.

Which are probably the most configurable and flexible.   These are super
easy to use, and allow you maximal control over your environment.

1) git clone https://github.com/apache/bigtop
2) cd bigtop-deploy/vm/vagrant/vagrant-puppet
3) Follow the directions in the README to create your hadoop cluster

You can look into the provision script to see how you can customize exactly
which components come (hbase,mahout,pig,....) installed in your
distribution.

Feel free to drop a line on the bigtop mailing list if you need any help
with getting them up and running.





On Sun, Jul 6, 2014 at 12:47 PM, Andre Kelpe <akelpe@concurrentinc.com>
wrote:

> We have a multi-vm or single-vm setup with apache hadoop, if you want to
> give that a spin:
> https://github.com/Cascading/vagrant-cascading-hadoop-cluster
>
> - André
>
>
> On Sun, Jul 6, 2014 at 9:05 AM, MrAsanjar . <afsanjar@gmail.com> wrote:
>
>> For my hadoop development and testing I use LXC (linux container) instead
>> of VM, mainly due to its light weight resource consumption. As mater of
>> fact as I am typing, my ubuntu system is automatically building a 6 nodes
>> hadoop cluster on my 16G labtop.
>> If you have an Ubuntu system you could install a fully configurable
>> Hadoop 2.2.0 single node or multi-node cluster in less then 10 minutes.
>> Here what you need to do:
>> 1) Install and learn Ubuntu Juju (shouldn't take an hour)- instructions :
>> https://juju.ubuntu.com/docs/getting-started.html
>> 2) there are two types hadoop charms:
>>      a) Single node for hadoop development :
>> https://jujucharms.com/?text=hadoop2-devel
>>       b) multi-node for testing  testing :
>> https://jujucharms.com/?text=hadoop
>> Let me know if you need more help
>>
>>
>> On Sun, Jul 6, 2014 at 7:59 AM, Marco Shaw <marco.shaw@gmail.com> wrote:
>>
>>> Note that the CDH link is for Cloudera which only provides Hadoop for
>>> Linux.
>>>
>>> HDP has "pre-built VMs" for both Linux and Windows hosts.
>>>
>>> You can also search for "HDInsight emulator" which runs on Windows and
>>> is based on HDP.
>>>
>>> Marco
>>>
>>> On Jul 6, 2014, at 12:38 AM, Gavin Yue <yue.yuanyuan@gmail.com> wrote:
>>>
>>> http://hortonworks.com/products/hortonworks-sandbox/
>>>
>>> or
>>>
>>> CDH5
>>>
>>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/DemoVMs/Cloudera-QuickStart-VM/cloudera_quickstart_vm.html
>>>
>>>
>>>
>>> On Sat, Jul 5, 2014 at 11:27 PM, Manar Elkady <m.elkady@fci-cu.edu.eg>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am a newcomer in using Hadoop, and I read many online tutorial to set
>>>> up Hadoop on Window by using virtual machines, but all of them link to old
>>>> versions of Hadoop virtual machines.
>>>> Could any one help me to find a Hadoop virtual machine, which include a
>>>> newer version of hadoop? Or should I do it myself from scratch?
>>>> Also, any well explained Hadoop installing tutorial and any other
>>>> helpful material are appreciated.
>>>>
>>>>
>>>> Manar,
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>
>>
>
>
> --
> André Kelpe
> andre@concurrentinc.com
> http://concurrentinc.com
>



-- 
jay vyas

Mime
View raw message