aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joshua Cohen <jco...@apache.org>
Subject Re: Populate DiscoveryInfo in Mesos
Date Wed, 30 Mar 2016 22:58:55 GMT
Job names are not unique though, what would happen if multiple jobs had the
same name (either across roles or across environments in the same role)?

On Wed, Mar 30, 2016 at 5:33 PM, Zhitao Li <zhitaoli.cs@gmail.com> wrote:

> Stephan,
>
> So I've managed to run the official Mesos DNS docker container
> <https://hub.docker.com/r/mesosphere/mesos-dns/> under the Aurora vagrant
> environment and get some SRV/A recorded pulled from Mesos master from
> Aurora.
>
> Because Mesos DNS uses 'name' field if set with some string manipulation,
> for the job 'vagrant/test/http_example_docker', my prototype generates
> these DNS records:
>
> A record: vagranttesthttp-exampled.twitterscheduler.mesos
> SRV record: _vagranttesthttp-exampled._tcp.twitterscheduler.mesos.
>
> If we want to make current prototype useful for Mesos DNS, I suggest we
> change the name field to job name, which would generate record like:
> A: http_example_docker.twitterscheduler.mesos
> SRV: _http_example_docker._tcp.twitterscheduler.slave.mesos
>
> I'll update my patch after getting some signal from you. Thanks.
>
> On Fri, Mar 25, 2016 at 1:49 PM, Zhitao Li <zhitaoli.cs@gmail.com> wrote:
>
> > Hi Stephan,
> >
> > Thanks for looking at that prototype patch.
> >
> > I'll update the patch with the review comments, and probably add a global
> > flag of "populate_discovery_info" to toggle this behavior.
> >
> > About the optional fields: I think it'll be hard to come up a good set of
> > rules applicable to all orgs using Aurora + Mesos, because cluster
> > management and service discovery stack could differ from org to org.
> >
> > In a recent Mesos work group, some experience folks (Jie Yu and Ben
> > Mahler) mentioned some ideas of *TaskInfoDecorator, *which is some
> > optional and configurable plugin on Aurora scheduler side to allow
> operator
> > to set additional fields before sending the message to Mesos. I like such
> > idea because it would enable Aurora users to experiment faster. Do you
> > think this is an interesting idea worth pursuing?
> >
> >
> > On Fri, Mar 25, 2016 at 1:42 PM, Erb, Stephan <
> Stephan.Erb@blue-yonder.com
> > > wrote:
> >
> >> I had a closer look at the Mesos documentation, and a design document
> >> might be unnecessary. Most of the values are optional. We can therefore
> >> leave them out until we have a proper usecase for them.
> >>
> >> I left a couple of comments in the review request.
> >> ________________________________________
> >> From: Zhitao Li <zhitaoli.cs@gmail.com>
> >> Sent: Tuesday, March 22, 2016 21:15
> >> To: dev@aurora.apache.org
> >> Subject: Re: Populate DiscoveryInfo in Mesos
> >>
> >> Hi Stephan,
> >>
> >> Sorry for the delay on follow up on this. I took a quick look at Aurora
> >> code, and it's actually quite easy to pipe this information to Mesos
> (see
> >> https://reviews.apache.org/r/45177/ for quick prototype).
> >>
> >> I'll take a stab to see how I can get Mesos-DNS to work with this
> >> prototype.
> >>
> >> IMO, if this is something the community is interested, the main
> questions
> >> would be 1) how various fields would be mapped in different Aurora
> usages,
> >> and 2) to which level should opt-in/opt-out configured for populating
> such
> >> information.
> >>
> >> I actually don't have too much insights on how these usage conventions
> >> would be set (through command line of scheduler or job configuration?)
> >>
> >> Do you think a design doc is the best action here, or a more involved
> >> questionnaire about which fields would be useful for community, or what
> >> value they should take?
> >>
> >> On Mon, Mar 7, 2016 at 1:00 AM, Erb, Stephan <
> Stephan.Erb@blue-yonder.com
> >> >
> >> wrote:
> >>
> >> > That sounds like a good idea! Great.
> >> >
> >> > If you go ahead with this, please be so kind and start by posting a
> >> short
> >> > design document here on mailinglist (similar to those here
> >> > https://github.com/apache/aurora/blob/master/docs/design-documents.md
> ,
> >> > but probably shorter).
> >> >
> >> > This will allow us to split the discussion of the design from
> discussing
> >> > the actual implementation. I believe this is necessary, as the
> >> > DiscoveryInfo protocol is quite flexible (
> >> >
> >>
> http://mesos.apache.org/documentation/latest/app-framework-development-guide/
> >> > ).
> >> >
> >> > Thanks,
> >> > Stephan
> >> >
> >> >
> >> > ________________________________________
> >> > From: Zhitao Li <zhitaoli.cs@gmail.com>
> >> > Sent: Monday, March 7, 2016 00:05
> >> > To: dev@aurora.apache.org
> >> > Subject: Populate DiscoveryInfo in Mesos
> >> >
> >> > Hi,
> >> >
> >> > It seems like Aurora does not populate the "discovery" field in either
> >> > TaskInfo or ExecutorInfo in mesos.proto
> >> > <
> >> >
> >>
> https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L438
> >> > >
> >> > .
> >> >
> >> > I'm considering adding this to support retrieving port map in Mesos
> >> > directly. This would enable us to discovery this information directly
> >> from
> >> > Mesos side, and also enables us to build one universal service
> discovery
> >> > solution for multiple frameworks including Aurora.
> >> >
> >> > If no objection, I'll create a JIRA ticket for this task.
> >> >
> >> > Thanks.
> >> > --
> >> > Cheers,
> >> >
> >> > Zhitao Li
> >> >
> >>
> >>
> >>
> >> --
> >> Cheers,
> >>
> >> Zhitao Li
> >>
> >
> >
> >
> > --
> > Cheers,
> >
> > Zhitao Li
> >
>
>
>
> --
> Cheers,
>
> Zhitao Li
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message