mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kone <vinodk...@gmail.com>
Subject Re: Slave<->Executor protocol instead of C++ driver + language bindings
Date Mon, 13 Jan 2014 03:25:37 GMT
Hey Vetoshkin,

Thanks for bringing up the question. As Dave alluded to we have discussed
the pros and cons of getting away from language bindings in the past.

One of the advantages of us proving language bindings and frameworks
linking to it is that we have to implement non-trivial scheduler/executor
specific logic in a single place, the driver (sched.cpp and exec.cpp
respectively). These drivers are not simply wrappers that call in to the
scheduler/executor after getting a message from Mesos.

For example the scheduler driver does authentication and mesos master
discovery on behalf of the scheduler. If we were to get rid of the driver,
every framework (or language) has to cook up their own solutions for these
in addition to handling the transport of the protocol messages, which makes
writing a new framework rather tedious.

That said the executor driver unlike the scheduler driver, has very little
logic in it besides wrapping the executor. So we could conceivably get rid
of the executor bindings in favor of providing pure language
implementations. This requires some (substantial?) refactoring of
libprocess (mesos's communication framework) to send/receive HTTP wrapped
protobuf messages. Do you mind creating a ticket to track this work?

HTH,


On Sat, Jan 11, 2014 at 7:01 PM, Dave Lester <dave@ischool.berkeley.edu>wrote:

> HI Vetoshkin,
>
> At face value, I think the goal of making language bindings unnecessary in
> favor of adopting a protocol makes sense -- I'd love to see Mesos expand
> its support for framework languages, and this could potentially help make
> that easier.
>
> If I remember correctly, Vinod (cc'd) was discussing a similar idea a few
> months back. Perhaps he'd like to weigh in.
>
> Dave
>
>
> On Thu, Jan 9, 2014 at 10:09 PM, Vetoshkin Nikita <
> nikita.vetoshkin@gmail.com> wrote:
>
>> Hi!
>> Have you ever considered providing a protocol for implementing Executor in
>> foreign language additionally to writing various bindings yourself?
>> I think that protocol isn't that hard as you already use protobuf to
>> exchange information, but I haven't dug deep in the sources. Documenting
>> wire protocol could open ways to implement Executors in pure python or
>> golang or any other language. It could provide some benefits:
>>   * in Python one could use gevent which is pretty popular
>>   * golang has it's own eventloop builtin
>>   * pure language binding could save a lot of trouble bridging with
>> unfriendly C++
>>   * building and using pure language client could be performed using
>> native
>> tools like `pip install` or `go get` without need to establish c++ dev
>> environment.
>> Going further you could open Framework<->Master protocol to allow crazy
>> stuff like using all fancy Netty, Akka, Finagle, whatever... network
>> tools,
>> frameworks and libraries to talk to Mesos.
>>
>> Sorry to bother if it's a wrong place or there's already an answer in the
>> wild. Thanks in advance!
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message