hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billie Rinaldi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15821) Move Hadoop YARN Registry to Hadoop Registry
Date Mon, 08 Oct 2018 17:32:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642205#comment-16642205
] 

Billie Rinaldi commented on HADOOP-15821:
-----------------------------------------

That's right, we don't currently have a service to clean up the registry. Cleanup is performed
by the client when an application is stopped and by the AM for individual containers.

The YarnRegistryAttributes are used by RegistryDNS to create DNS records. I agree that we
should consider generalizing the concepts and that this can wait for a future ticket.

Here are some notes on how RegistryDNS is using the attributes:
 * yarn:persistence is used to select which ServiceRecordProcessor is used. Container and
application both have ServiceRecordProcessors. Container is the main one being used by YARN
services. The Slider AM had used the application processor.
 * For ServiceRecords with yarn:persistence = container, the following records are created:
 $COMPONENT_INSTANCE_NAME.$APP_NAME.$USER.$DOMAIN -> $IP
 $COMPONENT_NAME.$APP_NAME.$USER.$DOMAIN -> $IP
 $ID.$DOMAIN -> $IP
 * Sources of the named variables (the "yarn-service" ZK node is used by YARN services but
any service name should work):
|$USER, $APP_NAME, $ID|ZK node /registry/users/$USER/services/yarn-service/$APP_NAME/components/$ID
(e.g. container ID)|
|$DOMAIN|Configuration hadoop.registry.dns.domain-name|
|$COMPONENT_INSTANCE_NAME|ServiceRecord description (e.g. regionserver-2)|
|$COMPONENT_NAME|ServiceRecord yarn:component (e.g. regionserver)|
|$IP|ServiceRecord yarn:ip|

> Move Hadoop YARN Registry to Hadoop Registry
> --------------------------------------------
>
>                 Key: HADOOP-15821
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15821
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 3.2.0
>            Reporter: Íñigo Goiri
>            Assignee: Íñigo Goiri
>            Priority: Major
>         Attachments: HADOOP-15821.000.patch, HADOOP-15821.001.patch, HADOOP-15821.002.patch,
HADOOP-15821.003.patch, HADOOP-15821.004.patch, HADOOP-15821.005.patch, HADOOP-15821.006.patch,
HADOOP-15821.007.patch, HADOOP-15821.008.patch
>
>
> Currently, Hadoop YARN Registry is in YARN. However, this can be used by other parts
of the project (e.g., HDFS). In addition, it does not have any real dependency to YARN.
> We should move it into commons and make it Hadoop Registry.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message