mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sam Taha <taha...@gmail.com>
Subject Application Dependency Managaement ala YARN
Date Tue, 22 Oct 2013 22:34:57 GMT
I am looking to implement a custom executor, but I do not want to require
users to distribute the JARs and other related resources associated with my
executor onto every slave node.

I see from the code that I can addUris and even ".tgz" that get downloaded
and extracted during the execution of the executor, but I am not clear if
this will be practical if my JARs and other resources needed by the java
executor will be in the 10s or 100s of megabytes.

It is important that I don't require the user to install anything on the
slave nodes other than the base Mesos package since my Framework
application is very dynamic and code jars can vary by job in most cases. So
I really want everything dynamically downloaded per job run and I would
also like the resources to be cached if they have not changed...etc.

Is there any plan to support something like what exist in YARN which deals
with these of lifecycle resource manage application dependencies across the
cluster? Here is theYARN overview of what I am kind of asking about:

http://hortonworks.com/blog/management-of-application-dependencies-in-yarn/

Thanks in advance.

Thanks,
Sam Taha

http://www.grandlogic.com

Mime
View raw message