spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tnachen <>
Subject [GitHub] spark pull request: [SPARK-6350][Mesos] Make mesosExecutorCores co...
Date Sat, 21 Mar 2015 07:16:40 GMT
Github user tnachen commented on the pull request:
    @sryza I believe this setting is a bit unique for Mesos. I think it's better to clarify
the terminology here, of what does framework and executor means for Mesos. 
    In mesos a scheduler is the process which registers with Master to receives offers, and
use these offers to launch tasks through the master and eventually the slave. An executor
is responsible for actually launching a task and reporting task statuses that is launched
on the slaves, which either is a custom executor that fine-grain mode uses or the default
mesos executor that coarse-grain mode uses. And a framework is referring to the overall scheduler
+ executor.
    So in the Mesos fine-grain case, we have an custom executor that is only launched once
per slave, and that running executor takes up some cpu/mem itself.
    IMO I think calling it spark.mesos.executor.cores is fine enough, I wouldn't put framework
in there as framework is not the right terminology.

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message