Return-Path: X-Original-To: apmail-mesos-user-archive@www.apache.org Delivered-To: apmail-mesos-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4E57B11AB4 for ; Thu, 17 Jul 2014 18:45:45 +0000 (UTC) Received: (qmail 78364 invoked by uid 500); 17 Jul 2014 18:45:45 -0000 Delivered-To: apmail-mesos-user-archive@mesos.apache.org Received: (qmail 78322 invoked by uid 500); 17 Jul 2014 18:45:45 -0000 Mailing-List: contact user-help@mesos.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@mesos.apache.org Delivered-To: mailing list user@mesos.apache.org Received: (qmail 78253 invoked by uid 99); 17 Jul 2014 18:45:45 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Jul 2014 18:45:45 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of tstclair@redhat.com designates 209.132.183.37 as permitted sender) Received: from [209.132.183.37] (HELO mx5-phx2.redhat.com) (209.132.183.37) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 17 Jul 2014 18:45:41 +0000 Received: from zmail16.collab.prod.int.phx2.redhat.com (zmail16.collab.prod.int.phx2.redhat.com [10.5.83.18]) by mx5-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s6HIjKk2029010 for ; Thu, 17 Jul 2014 14:45:20 -0400 Date: Thu, 17 Jul 2014 14:45:19 -0400 (EDT) From: Tim St Clair To: user@mesos.apache.org Message-ID: <788484981.10432936.1405622719978.JavaMail.zimbra@redhat.com> In-Reply-To: References: <-5601936662831814199@unknownmsgid> <757443116.8179328.1405362613263.JavaMail.zimbra@redhat.com> <1070308288.8427207.1405388686151.JavaMail.zimbra@redhat.com> <1930444636.8860963.1405433243564.JavaMail.zimbra@redhat.com> Subject: Re: Mesos language bindings in the wild MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_10432935_1197153732.1405622719977" X-Originating-IP: [10.5.82.11] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - GC35 (Linux)/8.0.6_GA_5922) Thread-Topic: Mesos language bindings in the wild Thread-Index: xBuASu30bs3hBS3nTIlNwe0YUfi6Sg== X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_10432935_1197153732.1405622719977 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Inline - ----- Original Message ----- > From: "Vladimir Vivien" > To: user@mesos.apache.org > Sent: Tuesday, July 15, 2014 1:34:37 PM > Subject: Re: Mesos language bindings in the wild > Hi all, > Apologies for being super late to this thread. To answer Niklas point at the > start of the thread: Yes, I am thrilled to contribute in anyway I can. The > project is moving forward and making progress (slower than I want, but > progress regardless). > Going Native > Implementing a native client for Mesos is an arduous process right now since > there's little doc to guide developers. Once I went through C++ code and a > few emails, it became easy (even easier than I thought). If the push is for > more native client, at some point we will need basic internals to be > documented. > Mesos-Certified > Maybe a Mesos test suite can be used to certify native clients. There are > tons of unit tests in the code that already validate the source code. Maybe > some of those test logic can be pulled out / copied into a small stand-alone > mesos test server that clients can communicate with to run a test suite > (just an idea). This along with some documentation would help with quality > of native clients. +1. > In or Out of Core > Having native clients source hosted in core would be great since all code > would be in one location. Go code can certainly co-exist a subproject in > Mesos. Go's build workflow can be driven by Make. Go's dependency management > can work with repo subdirectories (at least according to 'go help > importpath', I haven't tested that myself). But, as Tom pointed out, the > thing that raises a flag for me is project velocity. If author wants to move > faster or slower than Mesos release cycles, there's no way to do so once the > code is part of core. > Anyway, I have gone on long enough. Looking for ward to feedback. I usually don't tread here, but perhaps a git-submodule works in this narrow case. Thoughts? > On Tue, Jul 15, 2014 at 10:07 AM, Tim St Clair < tstclair@redhat.com > wrote: > > Tom - > > > I understand the desire to create bindings outside the core. The point I > > was > > trying to make earlier around version semantics and testing was to 'Hedge' > > the risk. It basically creates a contract between core & framework+bindings > > writers. > > > No one ever intends to break compatibility, but it happens all the time and > > usually in some very subtle ways at first. A great example of this is a > > patch I recently submitted to Mesos where the cgroup code was writing an > > extra < > but recent modifications would cause the cgroup code to fail. Very subtle, > > and boom-goes-the-dynamite. > > > Below was an email I sent a while back, that outlines a possible > > hedge/contract. Please let me know what you think. > > > -------------------------- > > > > > > > > Greetings! > > > > > > > > I've conversed with folks about the idea of having a more formalized > > > release > > > > and branching strategy, such that others who are downstream can rely on > > > > certain version semantics when planning upgrades, etc. This becomes > > > doubly > > > > important as we start to trend towards a 1.0 release, and folks will > > > depend > > > > heavily on it for their core infrastructure, and APIs (Frameworks, and > > > EC). > > > > > > > > Therefore, I wanted to propose a more formalized branching and release > > > > strategy, and see what others think. I slightly modified this pattern > > > from > > > > the Condor & Kernel projects, which have well established processes. > > > > > > > > ------------------------------ > > > > Basic Idea: > > > > > > > > 1.) Create 2 Main Branches (Stable/Devel-Master based) > > > > 2.) Devel releases are cadence/time based and lightly tested. > > > > 3.) Stable series only accepts bug fixes. Merge path for all bug fixes > > > > deemed worthy, are through the stable series up to master. > > > > 4.) @ some point devel goes through a *hardning phase* and becomes the > > > new > > > > stable. > > > > > > > > ------------------------------ > > > > Version Semantics: > > > > > > > > Major.Minor.Revision-PatchBuild > > > > > > > > Major: > > > > - Compatibility breakage (usually protocol or api shift), or enough > > > minors > > > > to justify change. > > > If there is a major version change it should be taken with care and notify > > downstream usually through the > > > mailing lists. > > > > > > > > Minor: > > > > - Devel (Odd) - 1.1.x > > > > - Stable (Even) - 1.0.x > > > > > > > > Revision: > > > > - Devel - Cadence # Some set of feature enhancements > > > > - Stable - Bug and security fixes only (Higher bar of entry) > > > > > > > > PatchBuild: > > > > - Upstream - Whoops our bad, we found a bug or two > > > > - Downstream - Back-port build variant. > > > > > > > > ------------------------------ > > > > Series/Branches: > > > > > > > > Development Series - (Odd Minor #'s): 1.1.x > > > > The development series branches/tags are cadence based, and come off of > > > > master. All new features are added to master. All bug fixes should be > > > > merged through the stable series into the master. It should be ok to > > > > introduce destabilizing features from time to time, provided its agreed > > > upon > > > > by a Sheppard. > > > > > > > > Stable Series - (Even Minor #'s): 1.0.x > > > > Stable series should *only contain* bug fixes. This way, downstream folks > > > > have a common understanding that behavior should be maintained. Should > > > > downstream folks wish to back-port features, they can do that at their > > > own > > > > risk. Every release of the stable series has some measure of quality > > > > then > > > > a +1. E.g. running some clusters for a period of time (X), > > > > > > > In this model, stable series should be "stable" for writers against the > > API(s). > > > > Transition from Devel-> Stable: > > > > After some point, the development series needs to go through a hardening > > > > phase. This could include static analysis + running on some production > > > > cluster for a period of time. Folks typically plan the transition around > > > a > > > > conference series in order to announce the cool new features. > > > + You could test the bindings during this phase ^ but for stable series > > they > > should just work. > > > > ------------------------------ > > > Cheers, > > > Tim > > > > From: "Tom Arnfeld" < tom@duedil.com > > > > > > > To: user@mesos.apache.org > > > > > > Sent: Tuesday, July 15, 2014 2:50:47 AM > > > > > > Subject: Re: Mesos language bindings in the wild > > > > > > Hey Tim, > > > > > > I can see your point, and am finding it hard to think of any compelling > > > arguments against the issue of fragmentation, but I do have a few > > > thoughts.p > > > > > > That said, I would strongly suggest taking ease-of-use and language > > > specific > > > code structures into consideration. A huge monolithic build system might > > > not > > > be a good thing either, if I'm not mistaken that's why twitter built > > > Pants. > > > > > > Spark is actually a great example here, it's going to be a huge pain to > > > publish PySpark to PYPI because of the way they structure the code, > > > unless > > > you force users to use a bleeding edge version of setuptools to be able > > > to > > > install the software. In the case of PySpark (and other libraries that > > > require compiled dependencies, see Hadoofus on github which I > > > collaborated > > > on this exact issue). It's a nightmare. Projects that work well with > > > python > > > setuptools are projects that are just python, from my experience. > > > > > > That said, it's only a nightmare when you *have* precompiled dependencies > > > that need to be part of the build process. This is no longer the case > > > with > > > the new mesos bindings, so why make it so hard? > > > > > > Take Go as another example (this is similar to installing pip > > > dependencies > > > from github too) - a user can simply plug in the path to a repository and > > > away they go. It's easy, and will rapidly speed up adoption IMO. This > > > isn't > > > something that can easily be done if it's not in it's own repo, and the > > > Mesos repository is pretty huge now. > > > > > > My opinion is largely from a users perspective. However, I would ask the > > > question - how often does the framework API change in such a way that it > > > breaks compatibility? Will there be a need to orchestrate releases among > > > 20 > > > language bindings to get a release of the core out, how often? Would it > > > be > > > easier for developers to implement a change and also make that change > > > across > > > all languages at the same time, is that even really going to happen? > > > > > > It's also worth considering release cycles, with all bindings being built > > > into the core, it requires them all the be release together (or it's a > > > git > > > tag pain). Given that lots of the bindings are going to be (and already > > > are) > > > community driven, and only a few people are in charge of the Mesos > > > release > > > cycle (taking at least a few weeks for a release to come out) the pace > > > for > > > each binding has to be the same, and there's no autonomy. > > > > > > My personal feeling is that develop user experience isn't thought about > > > enough is these sorts of situations, and not having a good experience > > > either > > > to use or work on the code is a pain and can slow down adoption. > > > > > > Would be interested to hear what you all think, or if you completely > > > disagree > > > :-) > > > > > > Tom. > > > > > > On Tuesday, 15 July 2014, Tim St Clair < tstclair@redhat.com > wrote: > > > > > > > So... your response basically capitulates to the fragmentation > > > > argument: > > > > > > > > > > "Yes we will have binding strewn about of questionable quality that > > > > may, > > > > or > > > > may not, work with core." > > > > > > > > > > The point that I'm trying to make is, fragmentation *is not* a good > > > > thing. > > > > > > > > > > -------------------------------------- > > > > > > > > > > Case in point - The Hadoop Ecosystem (fragmentation) > > > > > > > > > > In order for anyone to make a salient stack of any measure, vendors > > > > have > > > > to > > > > knit together components into a stack which can then be consumed by the > > > > masses. > > > > > > > > > > -------------------------------------- > > > > > > > > > > Counterpoint - Spark (curating) libraries > > > > > > > > > > Spark bundles 1st order interface libraries as part of a curated core. > > > > You > > > > are guaranteed that the core will inter-operate, and PySpark is given > > > > 1st > > > > class standing. > > > > > > > > > > -------------------------------------- > > > > > > > > > > This is a bad idea, unless there is a plan to hedge the risk. > > > > > > > > > > -Tim > > > > > > > > > > ----- Original Message ----- > > > > > > > > > > > From: "yifan" < myanata@msn.com > > > > > > > > > > > > To: user@mesos.apache.org > > > > > > > > > > > Sent: Monday, July 14, 2014 7:10:34 PM > > > > > > > > > > > Subject: Re: Mesos language bindings in the wild > > > > > > > > > > > > > > > > > > > > > > Hi Tim, > > > > > > > > > > > > > > > > > > > > > > I found that in zookeeper, they also separate the bindings from the > > > > > core. > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZKClientBindings > > > > > > > > > > > > > > > > > > > > > > So, IMHO, I think it should be the maintainer's responsibility to > > > > > keep > > > > > > > > > > > the binding in healthy state, with clear documentation of which > > > > > version > > > > > > > > > > > of the mesos core they supports. > > > > > > > > > > > > > > > > > > > > > > Yifan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 07/14/2014 11:30 AM, Tim St Clair wrote: > > > > > > > > > > > > So I fear the fragmentation that can occur if we provide native > > > > > > bindings > > > > > > > > > > > > outside of the core, unless there is some mechanism for testing, & > > > > > > a > > > > > > well > > > > > > > > > > > > established versioning scheme. > > > > > > > > > > > > > > > > > > > > > > > > IMHO, priority inversion on 'versioning' should come before > > > > > > bindings > > > > > > to > > > > > > > > > > > > ensure we adhere to policy. > > > > > > > > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > > > > > > > > > -Tim > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > > > > > > > >> From: "Tom Arnfeld" < tom@duedil.com > > > > > > > > > > > > >> To: dev@mesos.apache.org > > > > > > > > > > > >> Cc: user@mesos.apache.org > > > > > > > > > > > >> Sent: Friday, July 11, 2014 10:22:59 AM > > > > > > > > > > > >> Subject: Re: Mesos language bindings in the wild > > > > > > > > > > > >> > > > > > > > > > > > >> Very exciting. I'd vote +1 for splitting them out. Especially if > > > > > >> you > > > > > > > > > > > >> look at the common way of using Go imports, just stick the project > > > > > >> on > > > > > > > > > > > >> GitHub and import it directly using " github.com/mesos/mesos-go " > > > > > >> or > > > > > > > > > > > >> similar. > > > > > > > > > > > >> > > > > > > > > > > > >> I guess one argument is that you have more fragmentation of the > > > > > >> code > > > > > > > > > > > >> (e.g every library has it's own copy of the protos) but I'm not > > > > > >> sure > > > > > > > > > > > >> that's a bad thing. > > > > > > > > > > > >> > > > > > > > > > > > >> Just my two cents. Looking forward to this! > > > > > > > > > > > >> > > > > > > > > > > > >>> On 11 Jul 2014, at 16:59, Thomas Rampelberg < thomas@saunter.org > > > > > >>> > > > > > > >>> wrote: > > > > > > > > > > > >>> > > > > > > > > > > > >>> I've started preparing the python bindings to hopefully take this > > > > > > > > > > > >>> route ( https://reviews.apache.org/r/23224/ would love some > > > > > >>> reviews! > > > > > > > > > > > >>> ). In fact, there is already a native python implementation of > > > > > >>> both > > > > > > > > > > > >>> libprocess and the framework apis! ( > > > > > >>> https://github.com/wickman/pesos/ > > > > > > > > > > > >>> , https://github.com/wickman/compactor ). > > > > > > > > > > > >>> > > > > > > > > > > > >>> What are the benefits of bindings being part of the project > > > > > >>> source > > > > > > > > > > > >>> itself instead of having blessed implementations like > > > > > >>> mesos-python > > > > > > > > > > > >>> where the source and versioning becomes separate? I've been > > > > > >>> running > > > > > > > > > > > >>> into difficulties making automake and python's build tools play > > > > > >>> nicely > > > > > > > > > > > >>> together. It seems like there'd be more flexibility in general by > > > > > > > > > > > >>> splitting them out. > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>>> On Thu, Jul 10, 2014 at 3:57 PM, Niklas Nielsen < > > > > > >>>> niklas@mesosphere.io > > > > > >>>> > > > > > > > > > > > > >>>> wrote: > > > > > > > > > > > >>>> I just wanted to clarify - native, meaning _no_ dependency to > > > > > >>>> libmesos > > > > > > > > > > > >>>> and > > > > > > > > > > > >>>> native to its language (only Go, only Python and so on) i.e. use > > > > > >>>> the > > > > > > > > > > > >>>> low-level API. > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> Sorry for the confusion, > > > > > > > > > > > >>>> Niklas > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> > > > > > > > > > > > >>>>> On 10 July 2014 15:55, Dominic Hamon < dhamon@twopensource.com > > > > > >>>>> > > > > > > >>>>> wrote: > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> In my dream world, we wouldn't need any native bindings. I can > > > > > >>>>> imagine > > > > > > > > > > > >>>>> having example frameworks or starter frameworks that use the > > > > > >>>>> low-level > > > > > > > > > > > >>>>> API > > > > > > > > > > > >>>>> (the wire protocol with protocol buffers for message passing), > > > > > >>>>> but > > > > > > > > > > > >>>>> nothing > > > > > > > > > > > >>>>> like we have that needs C or JNI, etc. > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> On Thu, Jul 10, 2014 at 3:26 PM, Niklas Nielsen < > > > > > >>>>> niklas@mesosphere.io > > > > > > > > > > > > >>>>> wrote: > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>>> Hi all, > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>> I wanted to start a discussion around the language bindings in > > > > > >>>>>> the > > > > > > > > > > > >>>>>> wild > > > > > > > > > > > >>>>>> (Go, Haskell, native Python, Go, Java and so on) and possibly > > > > > >>>>>> get > > > > > >>>>>> to > > > > > >>>>>> a > > > > > > > > > > > >>>>>> strategy where we start bringing those into Mesos proper. As > > > > > >>>>>> most > > > > > > > > > > > >>>>>> things > > > > > > > > > > > >>>>>> points towards, it will probably make sense to focus on the > > > > > >>>>>> native > > > > > > > > > > > >>>>>> "bindings" leveraging the low-level API. To name one candidate > > > > > >>>>>> to > > > > > > > > > > > >>>>>> start > > > > > > > > > > > >>>>>> with, we are especially interested in getting Go native > > > > > >>>>>> support > > > > > >>>>>> in > > > > > > > > > > > >>>>>> Mesos > > > > > > > > > > > >>>>>> proper (and in a solid state). So Vladimir, we'd be super > > > > > >>>>>> thrilled > > > > > >>>>>> to > > > > > > > > > > > >>>>> start > > > > > > > > > > > >>>>>> collaborating with you on your current work. > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>> We are interested to hear what thoughts you all might have on > > > > > >>>>>> this. > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>> Thanks, > > > > > > > > > > > >>>>>> Niklas > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > Gu Yifan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Cheers, > > > > > > > > > > Timothy St. Clair > > > > > > > > > > Red Hat Inc. > > > > > > > > -- > > > Cheers, > > > Timothy St. Clair > > > Red Hat Inc. > > -- > Vladimir Vivien -- Cheers, Timothy St. Clair Red Hat Inc. ------=_Part_10432935_1197153732.1405622719977 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Inline - 

To= : user@mesos.apache.org
Sent: Tuesday, July 15, 2014 1:34:37 = PM
Subject: Re: Mesos language bindings in the wild

<= /div>
Hi all,
 Apologies for being super late to t= his thread.  To answer Niklas point at the start of the thread: Yes, I= am thrilled to contribute in anyway I can.  The project is moving for= ward and making progress (slower than I want, but progress regardless).&nbs= p;

Going Native
Implementing a nativ= e client for Mesos is an arduous process right now since there's little doc= to guide developers.  Once I went through C++ code and a few emails, = it became easy (even easier than I thought).  If the push is for more = native client, at some point we will need basic internals to be documented.=

Mesos-Certified
Maybe a Mesos test= suite can be used to certify native clients.  There are tons of unit = tests in the code that already validate the source code.  Maybe some o= f those test logic can be pulled out / copied into a small stand-alone meso= s test server that clients can communicate with to run a test suite (just a= n idea).  This along with some documentation would help with quality o= f native clients.

+1.&nbs= p;


In or Out of Core
Havin= g native clients source hosted in core would be great since all code would = be in one location. Go code can certainly co-exist a subproject in Mesos. &= nbsp;Go's build workflow can be driven by Make. Go's dependency management = can work with repo subdirectories (at least according to 'go help importpat= h', I haven't tested that myself).  But, as Tom pointed out, the thing= that raises a flag for me is project velocity.  If author wants to mo= ve faster or slower than Mesos release cycles, there's no way to do so once= the code is part of core. 

Anyway, I have go= ne on long enough.   Looking for ward to feedback.

I usually don't tread here, but perhaps a git-subm= odule works in this narrow case.  
Thoughts? 



On Tue, J= ul 15, 2014 at 10:07 AM, Tim St Clair <tstclair@redhat.com> wrote:
It basically cr= eates a contract between core & framework+bindings writers.  

No one ever intends to break compatibility, but= it happens all the time and usually in some very subtle ways at first. &nb= sp;A great example of this is a patch I recently submitted to Mesos wh= ere the cgroup code was writing an extra <<endln out.  Earlier v= ersions of the kernel had no issue with this, but recent modifications woul= d cause the cgroup code to fail.  Very subtle, and boom-goes-the-dynam= ite.

Below was an email I sent a while back, that = outlines a possible hedge/contract.  Please let me know what you think= .   

--------------------------
>
> Greetings= !
>
> I've conversed with folks about the idea of having a mor= e formalized release
> and branching strategy, such that others who a= re downstream can rely on
> certain version semantics when planning u= pgrades, etc.  This becomes doubly
> important as we start to t= rend towards a 1.0 release, and folks will depend
> heavily on it for= their core infrastructure, and APIs (Frameworks, and EC).
>
>= Therefore, I wanted to propose a more formalized branching and release
= > strategy, and see what others think.  I slightly modified this p= attern from
> the Condor & Kernel projects, which have well estab= lished processes.
>
> ------------------------------
> B= asic Idea:
>
> 1.) Create 2 Main Branches (Stable/Devel-Maste= r based)
> 2.) Devel releases are cadence/time based and lightly test= ed.
> 3.) Stable series only accepts bug fixes.  Merge path for = all bug fixes
> deemed worthy, are through the stable series up to m= aster.
> 4.) @ some point devel goes through a *hardning phase* and b= ecomes the new
> stable.
>
> ---------------------------= ---
> Version Semantics:
>
> Major.Minor.Revision-Patch= Build
>
> Major:
>  - Compatibility breakage (usual= ly protocol or api shift), or enough minors
>  to justify change= .


If there is a major versi= on change it should be taken with care and notify downstream usually throug= h the 

mail= ing lists. 


>   
> Minor:
>  - Devel (Odd) - 1.1.x >  - Stable (Even) - 1.0.x
>
> Revision:
> &n= bsp;- Devel - Cadence # Some set of feature enhancements
>  - St= able - Bug and security fixes only (Higher bar of entry)
>
> P= atchBuild:
>  - Upstream - Whoops our bad, we found a bug or tw= o
>  - Downstream - Back-port build variant.
>
> --= ----------------------------
> Series/Branches:
>
> Deve= lopment Series - (Odd Minor #'s): 1.1.x
> The development series bra= nches/tags are cadence based, and come off of
> master.  All new= features are added to master.  All bug fixes should be
> merged= through the stable series into the master.  It should be ok to
&g= t; introduce destabilizing features from time to time, provided its agreed = upon
> by a Sheppard.
>
> Stable Series - (Even Minor #'= s): 1.0.x
> Stable series should *only contain* bug fixes.  This= way, downstream folks
> have a common understanding that behavior s= hould be maintained.  Should
> downstream folks wish to back-por= t features, they can do that at their own
> risk.  Every release= of the stable series has some measure of quality > then
> a +1. =  E.g. running some clusters for a period of time (X),
>


In this model, stable series should= be "stable" for writers against the API(s).


> Transition from Devel-> Stable:> After some point, the development series needs to go through a harden= ing
> phase.  This could include static analysis + running on so= me production
> cluster for a period of time.  Folks typically = plan the transition around a
> conference series in order to announce= the cool new features.


+ Y= ou could test the bindings during this phase ^ but for stable series they s= hould just work.


> -----= -------------------------


Cheers,
Ti= m


From: "Tom Arnfeld" <tom@d= uedil.com>
To: user@mesos.a= pache.org
Sent: Tuesday, July 15, 2014 2:50:47 AM

Subject: Re: Mesos language bindings in the wild
<= div>
Hey Tim,

I can see your point, and am find= ing it hard to think of any compelling arguments against the issue of fragm= entation, but I do have a few thoughts.p

That said= , I would strongly suggest taking ease-of-use and language specific code st= ructures into consideration. A huge monolithic build system might not be a = good thing either, if I'm not mistaken that's why twitter built Pants.

Spark is actually a great example here, it's going to = be a huge pain to publish PySpark to PYPI because of the way they structure= the code, unless you force users to use a bleeding edge version of setupto= ols to be able to install the software. In the case of PySpark (and other l= ibraries that require compiled dependencies, see Hadoofus on github which I= collaborated on this exact issue). It's a nightmare. Projects that work we= ll with python setuptools are projects that are just python, from= my experience.

That said, it's only a nightmare w= hen you *have* precompiled dependencies that need to be part of the build p= rocess. This is no longer the case with the new mesos bindings, so why make= it so hard?

Take Go as another example (this is s= imilar to installing pip dependencies from github too) - a user can simply = plug in the path to a repository and away they go. It's easy, and will rapi= dly speed up adoption IMO. This isn't something that can easily be done if = it's not in it's own repo, and the Mesos repository is pretty huge now.

My opinion is largely from a users perspective. Howev= er, I would ask the question - how often does the framework API change in s= uch a way that it breaks compatibility? Will there be a need to orchestrate= releases among 20 language bindings to get a release of the core out, how = often? Would it be easier for developers to implement a change and also mak= e that change across all languages at the same time, is that even really go= ing to happen?

It's also worth considering re= lease cycles, with all bindings being built into the core, it requires= them all the be release together (or it's a git tag pain). Given that lots= of the bindings are going to be (and already are) community driven, and on= ly a few people are in charge of the Mesos release cycle (taking at least a= few weeks for a release to come out) the pace for each binding has to be t= he same, and there's no autonomy.

My personal feeling i= s that develop user experience isn't thought about enough is these sorts of= situations, and not having a good experience either to use or work on the = code is a pain and can slow down adoption.

Would be inte= rested to hear what you all think, or if you completely disagree :-)
<= div>
Tom.

On Tuesday, 15 July 2014, Tim St= Clair <tstclair@redhat.com> wrote:
So... your response basically c= apitulates to the fragmentation argument:  

"Yes we will have = binding strewn about of questionable quality that may, or may not, work wit= h core."

The point that I'm trying to make is, fragmentation *is no= t* a good thing.  

--------------------------------------
= Case in point - The Hadoop Ecosystem (fragmentation)

In order for a= nyone to make a salient stack of any measure, vendors have to knit together= components into a stack which can then be consumed by the masses.

= --------------------------------------
Counterpoint - Spark (curating) = libraries

Spark bundles 1st order interface libraries as part of a = curated core.  You are guaranteed that the core will inter-operate, an= d PySpark is given 1st class standing.

----------------------------= ----------

This is a bad idea, unless there is a plan to hedge the = risk.  

-Tim


----- Original Message -----
>= From: "yifan" <myanata@msn.com>
&= gt; To: user@mesos.apache.org
> Sent:= Monday, July 14, 2014 7:10:34 PM
> Subject: Re: Mesos language bind= ings in the wild
>
> Hi Tim,
>
> I found that i= n zookeeper, they also separate the bindings from the core.
>
&g= t; https://cwiki.apache.org/conflue= nce/display/ZOOKEEPER/ZKClientBindings
>
> So, IMHO, I th= ink it should be the maintainer's responsibility to keep
> the bindi= ng in healthy state, with clear documentation of which version
> of = the mesos core they supports.
>
> Yifan
>
>
= >
> On 07/14/2014 11:30 AM, Tim St Clair wrote:
> > So= I fear the fragmentation that can occur if we provide native bindings
= > > outside of the core, unless there is some mechanism for testing, = & a well
> > established versioning scheme.
> >
= > > IMHO, priority inversion on 'versioning' should come before bind= ings to
> > ensure we adhere to policy.
> >
> &g= t; Thoughts?
> >
> > -Tim
> >
> > > > ----- Original Message -----
> >> From: "Tom Arnf= eld" <tom@duedil.com>
> >>= ; To: dev@mesos.apache.org
> >>= Cc: user@mesos.apache.org
> >>= Sent: Friday, July 11, 2014 10:22:59 AM
> >> Subject: Re: Mes= os language bindings in the wild
> >>
> >> Very e= xciting. I'd vote +1 for splitting them out. Especially if you
> >= ;> look at the common way of using Go imports, just stick the project on=
> >> GitHub and import it directly using "github.com/mesos/mesos-go" or
> >> sim= ilar.
> >>
> >> I guess one argument is that you = have more fragmentation of the code
> >> (e.g every library ha= s it's own copy of the protos) but I'm not sure
> >> that's a = bad thing.
> >>
> >> Just my two cents. Looking f= orward to this!
> >>
> >>> On 11 Jul 2014, at = 16:59, Thomas Rampelberg <thomas@saunter.org= > wrote:
> >>>
> >>> I've started pr= eparing the python bindings to hopefully take this
> >>> ro= ute ( https://reviews.apache.o= rg/r/23224/ would love some reviews!
> >>> ). In fact, = there is already a native python implementation of both
> >>&g= t; libprocess and the framework apis! (https://github.com/wickman/pesos/
> >>> , https://github.com/wickman/compac= tor ).
> >>>
> >>> What are the benefit= s of bindings being part of the project source
> >>> itself= instead of having blessed implementations like mesos-python
> >&= gt;> where the source and versioning becomes separate? I've been running=
> >>> into difficulties making automake and python's build= tools play nicely
> >>> together. It seems like there'd be= more flexibility in general by
> >>> splitting them out. > >>>
> >>>
> >>>> On Thu= , Jul 10, 2014 at 3:57 PM, Niklas Nielsen <ni= klas@mesosphere.io>
> >>>> wrote:
> >&g= t;>> I just wanted to clarify - native, meaning _no_ dependency to li= bmesos
> >>>> and
> >>>> native to it= s language (only Go, only Python and so on) i.e. use the
> >>&= gt;> low-level API.
> >>>>
> >>>> = Sorry for the confusion,
> >>>> Niklas
> >>= >>
> >>>>
> >>>>> On 10 July= 2014 15:55, Dominic Hamon <dhamon@twopensour= ce.com> wrote:
> >>>>>
> >>>&g= t;> In my dream world, we wouldn't need any native bindings. I can imagi= ne
> >>>>> having example frameworks or starter frame= works that use the low-level
> >>>>> API
> >= ;>>>> (the wire protocol with protocol buffers for message pass= ing), but
> >>>>> nothing
> >>>>&g= t; like we have that needs C or JNI, etc.
> >>>>>
= > >>>>>
> >>>>>
> >>&= gt;>>
> >>>>> On Thu, Jul 10, 2014 at 3:26 PM, = Niklas Nielsen <niklas@mesosphere.io><= br> > >>>>> wrote:
> >>>>>
>= >>>>>> Hi all,
> >>>>>>
>= ; >>>>>> I wanted to start a discussion around the langua= ge bindings in the
> >>>>>> wild
> >>= >>>> (Go, Haskell, native Python, Go, Java and so on) and possi= bly get to a
> >>>>>> strategy where we start brin= ging those into Mesos proper. As most
> >>>>>> thi= ngs
> >>>>>> points towards, it will probably make= sense to focus on the native
> >>>>>> "bindings" = leveraging the low-level API. To name one candidate to
> >>>= ;>>> start
> >>>>>> with, we are especial= ly interested in getting Go native support in
> >>>>>= > Mesos
> >>>>>> proper (and in a solid state).= So Vladimir, we'd be super thrilled to
> >>>>> start=
> >>>>>> collaborating with you on your current w= ork.
> >>>>>>
> >>>>>> We= are interested to hear what thoughts you all might have on this.
> = >>>>>>
> >>>>>> Thanks,
>= >>>>>> Niklas
>
>
> --
> Gu = Yifan
>
>

--
Cheers,
Timothy St. Clair
Re= d Hat Inc.



--
=
Cheers,
Timothy St. Clair
Red Hat Inc.



--
Vlad= imir Vivien



--
Cheers,
Timothy St. Clair
Red Hat= Inc.
------=_Part_10432935_1197153732.1405622719977--