hama-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hama Wiki] Update of "GettingStartedMesos" by JeffFenchel
Date Tue, 03 Jun 2014 19:49:14 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.

The "GettingStartedMesos" page has been changed by JeffFenchel:
https://wiki.apache.org/hama/GettingStartedMesos

New page:
<<TableOfContents(5)>>

= Getting Started with Hama on Mesos =

== Requirements ==

In order to run Hama on Mesos it is required that Mesos already be installed on the cluster.
Instructions to set up Mesos may be found at the project website: http://mesos.apache.org/.

== Building ==

Be sure to build Hama with your version of Mesos specified: 

{{{
mvn clean install -Phadoop1 -Dhadoop.version=1.x.x -Dmesos.version=0.18.2
}}}

The groom servers will be set up by Mesos during execution of a job. In order for Mesos to
do this, you will need to upload a built version of Hama to a place where the Mesos slaves
can find it, such as the HDFS:

{{{
hadoop fs -put hama-7.0.0.tar.gz /hama-7.0.0.tar.gz
}}}

== Configuration ==

There are several properties that must be set in hama-site.xml:

{{{
<property>
  <name>bsp.master.TaskWorkerManager.class</name>
  <value>org.apache.hama.bsp.MesosScheduler</value>
  <description>
    This tells the scheduler to use Mesos 
    in order to execute tasks of each job
  </description>
</property>
<property>
  <name>hama.mesos.executor.uri</name>
  <value>hdfs://hdfs.name.node:port/hama.tar.gz</value>
  <description> 
     This is the URI of the Hama distribution. Upload this yourself. 
  </description>
</property>
<property>
  <name>bsp.tasks.maximum.total</name>
  <value>10</value>
  <description>
    This is an override for the total maximum tasks that may be run.
    The default behavior is to determine a value based on the available groom
    servers. However, if using Mesos, the groom servers are not yet allocated. 
    So, a value indicating the maximum number of slots available 
    in the cluster is needed. 
  </description>
</property>
<property>
  <name>hama.mesos.master</name>
  <value>local</value>
  <description>
    This is the address of the Mesos master instance. If you're using
    Zookeeper for master election, use the Zookeeper address here (i.e.,
    zk://zk.apache.org:2181/hadoop/mesos).
  </description>
</property>
}}}

Hama requires one cpu and memory defined by bsp.child.java.opts for each slot. This means
that a cluster with {{{bsp.tasks.maximum.total}}} set to {{{2}}} and {{{bsp.child.jova.opts}}}
set to {{{-Xmx1024m}}} will need at least {{{2}}} cpus and and {{{2048m}}} of memory.   

Manually distributing the configuration is not necessary. Hama and Mesos will distribute the
configuration and provide overrides where necessary. 

== Starting The BSPMaster ==
With Hama on Mesos you only need to set up and start the BSPMaster. After setting the configuration
the bsp master may be started:
{{{
% $HAMA_HOME/bin/hama-daemon.sh start bspmaster
}}}

== Testing ==

In order to test the setup, run an example from Hama:

{{{
$HAMA_HOME/bin/hama jar hama-examples-x.x.x.jar gen fastgen 100 10 randomgraph 2
}}}

Then verify that Mesos tasks are being created run. Do this by navigating to the Mesos status
page and clicking on the id of the Hama framework.  There should be tasks listed in either
the active or completed tasks section. 


Please email any questions to user@hama.apache.org

Mime
View raw message