aurora-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhitao Li (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (AURORA-1052) Populate Labels in TaskConfig
Date Wed, 20 Jan 2016 18:50:40 GMT

    [ https://issues.apache.org/jira/browse/AURORA-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108936#comment-15108936
] 

Zhitao Li edited comment on AURORA-1052 at 1/20/16 6:50 PM:
------------------------------------------------------------

We are working on a "sidecar" agent next to every Mesos agent, to perform some responsibilities
which is difficult to model in Mesos tasks (caching metadata, configuration and translation
data). The agent needs to resolve the job name and task id of all tasks running on the machine,
and we want to avoid polling Aurora scheduler every time, so inspecting "state.json" on Mesos
agent sounds a natural choice.

Our Aurora jobs are created by another service (which uses thrift to talk to Aurora scheduler)
and we can have multiple jobs mapped out of one logical service, so it would very flexible
if we can a mechanism to specify labels which could be seen at Mesos level.

bq. Zameer suggested that it might be useful to set the Aurora jobkey parts (environment,
role, ..) as Mesos labels. Would that be needed?

It's unclear right now whether we'll need these labels, and the side effect of silently turning
on many labels may outweigh things. We could add them in additional patches if it turns out
we need them. Also, our jobs are generated at thrift level rather than using aurora client,
so systematic and consistent generation of labels are not difficult for us, so the decision
should be made on what the entire community using client really wants.

bq. In order to prevent name clashes, the patch introduces a `METADATA_LABEL_PREFIX`. Maybe
this should be configurable?

1. Given that Aurora scheduler is the only process which generates Mesos {{Label}}s, I don't
feel like name clash is really a risk.
2. If the intent is to avoid auto-generated labels to clash with user generated ones, Docker
established some convention for [label namespaces|https://docs.docker.com/engine/userguide/labels-custom-metadata/#label-keys-namespaces],
which seems pretty reasonable to me. So all auto-generated labels would be prefixed by "org.apache.aurora."
while user generated labels would be prefixed by their org name.
3. If we need to separate out prefix and metadata key, we could consider adding a new optional
{{label_prefix}} field in the {{Metadata}} thrift  struct.


was (Author: zhitao):
We are working on a "sidecar" agent next to every Mesos agent, to perform some responsibilities
which is difficult to model in Mesos tasks (caching metadata, configuration and translation
data). The agent needs to resolve the job name and task id of all tasks running on the machine,
and we want to avoid polling Aurora scheduler every time, so inspecting "state.json" on Mesos
agent sounds a natural choice.

Our Aurora jobs are created by another service (which uses thrift to talk to Aurora scheduler)
and we can have multiple jobs mapped out of one logical service, so it would very flexible
if we can a mechanism to specify labels which could be seen at Mesos level.

bq. Zameer suggested that it might be useful to set the Aurora jobkey parts (environment,
role, ..) as Mesos labels. Would that be needed?

It's unclear right now whether we'll need these labels, and the side effect of silently turning
on many labels may outweigh things. We could add them in additional patches if it turns out
we need them. Also, like I explained, our jobs are generated at thrift level rather than using
aurora client, so systematic and consistent generation of labels are not difficult for us,
and the decision should be made on what community using Aurora client really wants.

bq. In order to prevent name clashes, the patch introduces a `METADATA_LABEL_PREFIX`. Maybe
this should be configurable?

1. Given that Aurora scheduler is the only process which generates Mesos {{Label}}s, I don't
feel like name clash is really a risk.
2. If the intent is to avoid auto-generated labels to clash with user generated, Docker established
some convention for [label namespaces|https://docs.docker.com/engine/userguide/labels-custom-metadata/#label-keys-namespaces],
which seems pretty reasonable to me. So all auto-generated labels are prefixed by "org.apache.aurora."
while user generated labels are prefixed by their org name.
3. If we need to separate out prefix and metadata key, we could consider adding a new optional
{{label_prefix}} field in the {{Metadata}} thrift  struct.

> Populate Labels in TaskConfig
> -----------------------------
>
>                 Key: AURORA-1052
>                 URL: https://issues.apache.org/jira/browse/AURORA-1052
>             Project: Aurora
>          Issue Type: Story
>          Components: Scheduler
>            Reporter: Stephan Erb
>            Priority: Minor
>              Labels: newbie
>
> Mesos has introduced labels on tasks (MESOS-2120). These correspond to what Aurora calls
metadata. 
> We should therefore set task labels according to our metadata information.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message