Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 3DA17200D28 for ; Mon, 23 Oct 2017 21:48:10 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 3C4E91609E0; Mon, 23 Oct 2017 19:48:10 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 950181609DF for ; Mon, 23 Oct 2017 21:48:08 +0200 (CEST) Received: (qmail 23037 invoked by uid 500); 23 Oct 2017 19:48:07 -0000 Mailing-List: contact user-help@mesos.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@mesos.apache.org Delivered-To: mailing list user@mesos.apache.org Received: (qmail 23027 invoked by uid 99); 23 Oct 2017 19:48:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 Oct 2017 19:48:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 7DACE1A0B77 for ; Mon, 23 Oct 2017 19:48:06 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.379 X-Spam-Level: ** X-Spam-Status: No, score=2.379 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id XY8mjc-eItGb for ; Mon, 23 Oct 2017 19:48:03 +0000 (UTC) Received: from mail-qt0-f177.google.com (mail-qt0-f177.google.com [209.85.216.177]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 8880D5FC1C for ; Mon, 23 Oct 2017 19:48:03 +0000 (UTC) Received: by mail-qt0-f177.google.com with SMTP id k31so27703306qta.6 for ; Mon, 23 Oct 2017 12:48:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Hcx7lYOHKchW4nmchSWom2OAPqpWnpVPSAD6wmIbM98=; b=IqBGB925mESSZSC+gUWdOR1qXQLvcDbew07Pos1iNqsEd3fANmErR6yq6VH0QPE0Ur JMt7OYg5xU45+2C76xjzTYdCn3IiprJdSHno+V9nb9CJhwHSfhX2FeopL9ud4QbeZ9Zs VYBvkGB5MB9q5udFYTFJcb/a+nHvPHonu816/aXeuvdXqXRY/YCo/tDhbzolyUoh2kcD Fgcg51aVC4j1rrBWLLCl2rkOvCCgu9uoa41/Ea1ap3Y4r1kFcQZvUyx2fE+CwDcuNL6z qfVV/jWjBXtg1tecbGeSMUkP0kIWnRIS5wYLa1FHl6GibAHTVWjr3VUIukPIntLdUKGL sMNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Hcx7lYOHKchW4nmchSWom2OAPqpWnpVPSAD6wmIbM98=; b=AIs+aMejb++75KB89N9iVw8IU3PLU+hoGn+KuwJgOM+qTKuLDHe+XB5sG9TKxFOnJg hVO6d9wN8+U6uZ+4dvhT3x9vd9CN5o2z4tnbjt3RA8ehS1MK7FYTlJu7pNc0nZkqebtH j//nZsb5Kkh8F1KhvAvPrn2dtH4yXCnIA3o+68QtEkbfFhM5yc/uBjTY2rYXuDhcq2Ym jJl8ihhqkzlThgoCiHpgWCJ5nHNqf0wODJbV67DG4ypAdZ9ikWYh3EHsyBvBHJUmLqT/ kZk0lVA5Gg5+9mo2BzIJbu1Ikwu8pQkmOYwbdlhUxbzKgb8f6HScZPZPvrMfwScRfLTh MGKg== X-Gm-Message-State: AMCzsaVY/LY4YT8BljlPvgxH9Fd8b8aUMHjErOGROHuGTqpyx4Orp1Tm V8IPJld5RlJpfw9sC/1+vNJsUV2J37p7HGGn6PuMxg== X-Google-Smtp-Source: ABhQp+RPM6HgS738RdCFs0PrKEWL1XVaCk0E3zm2pR74g7HqG1BqkYozHWt1AhTzCmfFRoTKEM2Gs2tiAelhs9JU+WI= X-Received: by 10.200.52.235 with SMTP id x40mr21318847qtb.135.1508788082631; Mon, 23 Oct 2017 12:48:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.97.7 with HTTP; Mon, 23 Oct 2017 12:47:42 -0700 (PDT) In-Reply-To: References: From: Harpreet Singh Date: Mon, 23 Oct 2017 12:47:42 -0700 Message-ID: Subject: Re: Mesos acceptedResourceRoles slave_public To: user@mesos.apache.org Content-Type: multipart/alternative; boundary="001a1149e34cb4e79f055c3c197d" archived-at: Mon, 23 Oct 2017 19:48:10 -0000 --001a1149e34cb4e79f055c3c197d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Marc The slave_public is a DC/OS construct and has no inherent significance in Mesos/Marathon. My guess is that you launched Marathon with default configuration where --mesos_role flag defaults to None ( ref https://mesosphere.github.io/marathon/docs/command-line-flags.html) . Hence Marathon does not recognize the role slave_public and hence might be complaining. Instead of explaining how to configure marathon/mesos to use a slave_public role I must first ask *What problem are you trying to actually solve?* If the answer to that question is that you want to launch certain apps on certain nodes in the cluster instead of others due to some reason then do explore the possibility of using attributes/constraints (ref: https://mesosphere.github.io/marathon/docs/constraints.html). Let me know if you have questions configuring attributes on mesos and then using constraints on Marathon. But if you are already familiar with constraints and you consciously made a choice of not using them and using roles for a particular reason then you would have to do a couple of things to make it work : 1) Make resource reservation for the desired role in Mesos so that Marathon can be offered the resources with that role 2) Configure Marathon to use that particular resource role If you have not done either of these steps it would not work. Expanding on these two points in above detail : *1) Making resource reservation in Mesos* I would once again stress that make sure you cannot solve this problem with attributes because I=E2=80=99m of the opinion that there is value to not ha= ving to statically partition your cluster resources unless there is no better way out. If you are sold on the idea on using static resource reservations then you can configure resources on your agent node using the --resources flag (ref: http://mesos.apache.org/documentation/latest/reservation/) and carve out resources for the desired role (Ref http://mesos.apache.org/documentation/latest/roles/). I purposely did not use the term slave_publicfor role because you can name it to whatever you desire. Also note that --roles flag for Mesos master is deprecated no longer required to be specified. Lets assume you used custom_role as a the role. *2) Configuring Marathon to use the custom role* Please note that even though Mesos now supports a single framework accepting offers from mutiple roles ( https://issues.apache.org/jira/browse/MESOS-1763), Marathon has not caught upto this (https://jira.mesosphere.com/browse/MARATHON-2775) yet. So as of today Marathon can only get offers for one customer role and unreserved (*) resources. - Configuring --mesos_role flag Unless you specify --mesos_role custom_role Marathon is only going to accept offers from * resources. If you specify --mesos_role custom_role apps launched on Marathon would be able to accept offers from both * as well as custom_role. - Configuring --default_accepted_resource_roles flag Another important flag to be aware of here is --default_accepted_resource_roles. This flag dictates the deafult for "acceptedResourceRoles" attribute as a comma-separated list of strings. Any app definition which does not specify this attribute explicitly uses this value for launching new tasks. So you can use the correct mix of --mesos_role and --default_accepted_resource_roles to get desired behaviour. e.g. : =E2=80=94mesos_role =E2=80=94default_accepted_resource_roles acceptedResour= ceRoles resources used * custom_role * or custom_role custom_role *,custom_role * or custom_role custom_role * custom_role custom_role You can extrapolate the examples from the table above to come up with all possibilities and use the one that fits your needs. Let me know if you have followup questions. Cheers Harpreet On Fri, Oct 20, 2017 at 2:00 PM, Marc Roos wrote= : > Thanks :) But I am runing a test setup of 3 nodes on CentOS7, should > have written this with first post. > > Linux m01 3.10.0-693.2.2.el7.x86_64 > mesos-1.4.0-2.0.1.x86_64 > marathon-1.4.8-1.0.660.el7.x86_64 > containernetworking-cni-0.5.1-1.el7.x86_64 > mesosphere-zookeeper-3.4.6-0.1.20141204175332.centos7.x86_64 > chronos-2.5.0-0.1.20170816233446.el7.x86_64 > > > > > > -----Original Message----- > From: Bruce Campbell [mailto:Bruce.Campbell@microsoft.com] > Sent: vrijdag 20 oktober 2017 18:41 > To: user@mesos.apache.org > Subject: RE: Mesos acceptedResourceRoles slave_public > > In acs engine if you define a ports list, the agent will be deployed as > slave_public. > > Example of a public linux agent: > > "agentPoolProfiles": [ > { > "name": "wpub6", > "count": 3, > "vmSize": "Standard_DS2_v2", > "dnsPrefix": "wpub6", > "ports": [ > 80, > 443, > 8080, > 3389 > ], > "customNodeLabels" : { > "foo" : "bar", > "att1" : "value1" > } > }, > > > }, > > -----Original Message----- > From: Marc Roos [mailto:M.Roos@f1-outsourcing.eu] > Sent: Friday, October 20, 2017 9:29 AM > To: user > Subject: Mesos acceptedResourceRoles slave_public > > > I added "acceptedResourceRoles": ["slave_public"] to a json application > config in marathon. And it complains that the agents/slaves are not > configured for this. I see some manuals on how to reconfigure via dc/os. > > I just wondered could this be an easy setting like something like this > > echo "appc,docker" > /etc/mesos-slave/image_providers > > > > > > =E2=80=8B --001a1149e34cb4e79f055c3c197d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Hi Marc

The slave_publ= ic is a DC/OS construct and has no inherent significance in Mesos/Ma= rathon. My guess is that you launched Marathon with default configuration w= here --mesos_role flag defaults to None ( ref h= ttps://mesosphere.github.io/marathon/docs/command-line-flags.html) . He= nce Marathon does not recognize the role slave_public and hence might be complaining.

Instead of explaining how to co= nfigure marathon/mesos to use a slave_public role I= must first ask What problem are you trying to actually solve?

If the answer to that question = is that you want to launch certain apps on certain nodes in the cluster ins= tead of others due to some reason then do explore the possibility of using = attributes/constraints (ref: https://mesosphere.github.io/marathon/docs/constr= aints.html). Let me know if you have questions configuring attributes = on mesos and then using constraints on Marathon.

But if you are already familiar= with constraints and you consciously made a choice of not using them and u= sing roles for a particular reason then you would have to do a couple of th= ings to make it work :

1) Make resource reservation fo= r the desired role in Mesos so that Marathon can be offered the resources w= ith that role
2) Configure Marathon to use that particular resource role=

If you have not done either of = these steps it would not work.

Expanding on these two points i= n above detail :

1) Making resource rese= rvation in Mesos
I would once again stress that make sure you c= annot solve this problem with attributes because I=E2=80=99m of the opinion= that there is value to not having to statically partition your cluster res= ources unless there is no better way out. If you are sold on the idea on us= ing static resource reservations then you can configure resources on your a= gent node using the --resources flag (ref: http://mesos= .apache.org/documentation/latest/reservation/) and carve out resources = for the desired role (Ref http://mesos.apache.org/documentation/latest/roles/). I= purposely did not use the term slave_publicfor rol= e because you can name it to whatever you desire. Also note that --roles flag for Mesos master is deprecated no longer requir= ed to be specified. Lets assume you used custom_role as a the role.

2) Configuring Marathon= to use the custom role
Please note that even though Mesos now = supports a single framework accepting offers from mutiple roles (https://issues.apache.or= g/jira/browse/MESOS-1763), Marathon has not caught upto this (https://jira.mesosphe= re.com/browse/MARATHON-2775) yet. So as of today Marathon can only get = offers for one customer role and unreserved (*) res= ources.

  • Configuring --mesos_role flag
    Unless you specify --mesos_role custom_role Marathon is only going to accept offers from * resources. If you specify --mesos_role custom_role apps launched on Marathon would be able to accept offers from both * as well as custom_role.
  • Configuring --default_accep= ted_resource_roles flag
    Another important flag to be aware of her= e is --default_accepted_resource_roles. This flag d= ictates the deafult for "acceptedResourceRoles"<= /code> attribute as a comma-separated list of strings. Any app definition w= hich does not specify this attribute explicitly uses this value for launc= hing new tasks.

So you can use the correct mix = of --mesos_role and --default_acce= pted_resource_roles to get desired behaviour. e.g. :

=E2=80=94mesos_role =E2=80=94default_accepted_resource_roles acceptedResourceRoles resources used
<unspecified> <unspecified> <unspecified> *
custom_role <unspecified> <unspecified> * or custom_role
custom_role *,custom_role <unspecified> * or custom_role
custom_role * custom_role custom_role

You can extrapolate the example= s from the table above to come up with all possibilities and use the one th= at fits your needs.

Let me know if you have followu= p questions.

Cheers
Harpreet

On Fri, Oct 20, 2017 at 2:00 PM= , Marc Roos <M.Roos@f1-outsourcing.eu> wrote:


Thanks :) But I am runing a test setup of 3 nodes on CentOS7, should
have written this with first post.

Linux m01 3.10.0-693.2.2.el7.x86_64
mesos-1.4.0-2.0.1.x86_64
marathon-1.4.8-1.0.660.el7.x86_64
containernetworking-cni-0.5.1-1.el7.x86_64
mesosphere-zookeeper-3.4.6-0.1.20141204175332.centos7.x86_64
chronos-2.5.0-0.1.20170816233446.el7.x86_64
=





-----Original Message-----
From: Bruce Campbell [mailto:Bruce.Campbell@microsoft.com]
Sent: vrijdag 20 oktober 2017 18:41
To: user@mesos.a= pache.org
Subject: RE: Mesos acceptedResourceRoles slave_public

In acs engine if you define a ports list, the agent will be deployed as
slave_public.

Example of a public linux agent:

=C2=A0 =C2=A0"agentPoolProfiles": [
=C2=A0 =C2=A0 =C2=A0 {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 "name": "wpub6",
=C2=A0 =C2=A0 =C2=A0 =C2=A0 "count": 3,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 "vmSize": "Standard_DS2_v2"= ,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 "dnsPrefix": "wpub6",
=C2=A0 =C2=A0 =C2=A0 =C2=A0 "ports": [
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A080,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0443,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A08080,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A03389
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ],
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "customNodeLabels" : {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "foo" : "bar"= ,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "att1" : "value1&q= uot;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 =C2=A0 },


},

-----Original Message-----
From: Marc Roos [mailto:M.Roos@f1-outsourcing.eu]
Sent: Friday, October 20, 2017 9:29 AM
To: user <use= r@mesos.apache.org>
Subject: Mesos acceptedResourceRoles slave_public


I added "acceptedResourceRoles": ["slave_public"] to a = json application
config in marathon. And it complains that the agents/slaves are not
configured for this. I see some manuals on how to reconfigure via dc/os.
I just wondered could this be an easy setting like something like this

echo "appc,docker" > /etc/mesos-slave/image_providers





=E2= =80=8B
--001a1149e34cb4e79f055c3c197d--