hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-913) Add a way to register long-lived services in a YARN cluster
Date Thu, 25 Sep 2014 20:32:34 GMT

     [ https://issues.apache.org/jira/browse/YARN-913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Steve Loughran updated YARN-913:
    Attachment: YARN-913-010.patch

This patch doesn't look at why yesterday's jenkin tests failed, so if they are due to these
changes, those changes won't have been fixed.

Key changes are due to experience implementing a (not in this patch) read only REST view.
# renamed fields in the {{ServiceRecord}} because Jersey ignores {{@JsonProperty}} annotations
giving fields specific names. So no {{yarn:id}} {{yarn:persistence}} in the JSON; fields called
{{yarn_id}} and {{yarn_persistence}} instead.
# Specific exception {{NoRecordException}} to differentiate "could not resolve a node as there
isn't any entry with the header used to identify service records from {{InvalidRecordException}}
which is only triggered on parse problems.
# added a lightweight {{list()}} operation that only returns the child paths; the original
{{list(path) -> List<RegistryPathStatus>}} renamed to {{listFull}}. 

There's a CLI client for this being written; it'll help validate the API & identify any
further points for tuning

> Add a way to register long-lived services in a YARN cluster
> -----------------------------------------------------------
>                 Key: YARN-913
>                 URL: https://issues.apache.org/jira/browse/YARN-913
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: api, resourcemanager
>    Affects Versions: 2.5.0, 2.4.1
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: 2014-09-03_Proposed_YARN_Service_Registry.pdf, 2014-09-08_YARN_Service_Registry.pdf,
RegistrationServiceDetails.txt, YARN-913-001.patch, YARN-913-002.patch, YARN-913-003.patch,
YARN-913-003.patch, YARN-913-004.patch, YARN-913-006.patch, YARN-913-007.patch, YARN-913-008.patch,
YARN-913-009.patch, YARN-913-010.patch, yarnregistry.pdf, yarnregistry.tla
> In a YARN cluster you can't predict where services will come up -or on what ports. The
services need to work those things out as they come up and then publish them somewhere.
> Applications need to be able to find the service instance they are to bond to -and not
any others in the cluster.
> Some kind of service registry -in the RM, in ZK, could do this. If the RM held the write
access to the ZK nodes, it would be more secure than having apps register with ZK themselves.

This message was sent by Atlassian JIRA

View raw message