Mailing-List: contact dev-help@singa.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@singa.incubator.apache.org
Date: Wed, 14 Oct 2015 04:43:05 +0000 (UTC)
From: "Anh Dinh (JIRA)" <jira@apache.org>
To: dev@singa.incubator.apache.org
Message-ID: <JIRA.12837580.1434180744000.32694.1444797785697@Atlassian.JIRA>
In-Reply-To: <JIRA.12837580.1434180744000@Atlassian.JIRA>
References: <JIRA.12837580.1434180744000@Atlassian.JIRA>
 <JIRA.12837580.1434180744377@arcas>
Subject: [jira] [Updated] (SINGA-11) Start SINGA using Mesos
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/SINGA-11?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Anh Dinh updated SINGA-11:
--------------------------
    Description: 
Apache Mesos is a fine-grained cluster management framework which enables resource sharing in the
same cluster. Mesos abstracts out the physical configurations of cluster nodes, and presents
resources to the users in the form of "offers". SINGA uses Mesos for two purposes:

1. To acquire necessary resources for training the model.
2. To launch and monitor progress of the training task.

To this end, we implement a "SINGA Scheduler" which interacts with Mesos master. The scheduler
assumes that SINGA has been installed  at the Mesos slave nodes. The scheduler is called when the
user wants to start a new SINGA job, and it performs the following steps:

Step 1. Read the job configuration file to determine necessary resources in terms of CPUs, memory and storage.
Step 2. Wait for resource offers from the Mesos master.
Step 3. Determine if the offers meet the requirement of resources.
Step 4. Prepare the task to launch at each slave:
			+ Deliver the job configuration file to the slave node.
			+ Specify the command to run on the	slave: "singa -conf ./job.conf"
Step 5: Launch and monitor the progress

For step 3, we currently implement a simple scheme: the number of CPUs offered by each Mesos slave
exceed the total number of SINGA worker and SINGA server per process. In other words, each Mesos
slave must be able to run the entire worker group or server group.

For step 4, we currently relies on HDFS to deliver the configuration file to each slave.
Particularly, we write the file to a known directory (different for each job) on HDFS and ask the
slave to use its Fetcher utility to download the file before executing the task.

We will create a README.md file explaining the steps.

  was:
Mesos helps to mange resources in large clusters. 
This ticket is an initial integration of SINGA with Mesos, which aims to simply start SINGA through Mesos and run multiple SINGA tasks in the same cluster.
The fully integration should include,

1. start SINGA by Mesos, including requesting processes, memory, CPU, etc.
2. detect failures and recovery through Mesos
3. TBD.


> Start SINGA using Mesos
> -----------------------
>
>                 Key: SINGA-11
>                 URL: https://issues.apache.org/jira/browse/SINGA-11
>             Project: Singa
>          Issue Type: New Feature
>            Reporter: wangwei
>            Assignee: Anh Dinh
>
> Apache Mesos is a fine-grained cluster management framework which enables resource sharing in the
> same cluster. Mesos abstracts out the physical configurations of cluster nodes, and presents
> resources to the users in the form of "offers". SINGA uses Mesos for two purposes:
> 1. To acquire necessary resources for training the model.
> 2. To launch and monitor progress of the training task.
> To this end, we implement a "SINGA Scheduler" which interacts with Mesos master. The scheduler
> assumes that SINGA has been installed  at the Mesos slave nodes. The scheduler is called when the
> user wants to start a new SINGA job, and it performs the following steps:
> Step 1. Read the job configuration file to determine necessary resources in terms of CPUs, memory and storage.
> Step 2. Wait for resource offers from the Mesos master.
> Step 3. Determine if the offers meet the requirement of resources.
> Step 4. Prepare the task to launch at each slave:
> 			+ Deliver the job configuration file to the slave node.
> 			+ Specify the command to run on the	slave: "singa -conf ./job.conf"
> Step 5: Launch and monitor the progress
> For step 3, we currently implement a simple scheme: the number of CPUs offered by each Mesos slave
> exceed the total number of SINGA worker and SINGA server per process. In other words, each Mesos
> slave must be able to run the entire worker group or server group.
> For step 4, we currently relies on HDFS to deliver the configuration file to each slave.
> Particularly, we write the file to a known directory (different for each job) on HDFS and ask the
> slave to use its Fetcher utility to download the file before executing the task.
> We will create a README.md file explaining the steps.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)