singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wang Wei <wang...@comp.nus.edu.sg>
Subject Mesos Vs YARN?
Date Fri, 24 Apr 2015 14:52:28 GMT
Hi all,

Attached are the reports from Anh and Wangsheng on Mesos and YARN
respectively.

One purpose of using Mesos or YARN is to help implement fault-tolerance in
Singa. Mesos (YARN) can detect the failure of one executor (container) and
launch a new one. It makes the fault-tolerance easier.

There are many discussions on comparing YARN and Mesos, e.g.,
http://www.quora.com/How-does-YARN-compare-to-Mesos.

They provide almost the same functionality, i.e., resource management and
scheduling. Hence it is hard to select one over the other.* If you have any
experience on using these two software, please give us some suggestions.*

I list some differences here:

1. Both systems can use a script written by us to launch Singa on one node.
Since Mesos is written in C (C++), it provides one more method to launch
Singa. In specific, we inherit the Executor of Mesos and override the
LaunchTask() to insert Singa's main entrance code. Then Singa is launched
when Mesos calls the Executor's LaunchTask() function.

2. Mesos uses the same master(s) to connect with the executors for all
applications, while YARN launches the Application Master in one container
for each application. The workload of  Mesos' master would then be larger
than that of the resource manager of YARN, and may be overloaded if there
are too many applications in a large cluster.


Thanks.

Regards,
Wei​
 Mesos.pdf
<https://docs.google.com/file/d/0B_qmdIPX0hkSS2hmOFJ2RTV1VGEzeXl3ZnpSYVFxZVRGU2lF/edit?usp=drive_web>
​​
 YARN.pptx
<https://docs.google.com/file/d/0B_qmdIPX0hkSNm9tWi00SzIzV0NTQ253ZkMwZmxOSmNJNFlV/edit?usp=drive_web>
​

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message