hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-5638) Introduce a collector timestamp to uniquely identify collectors creation order in collector discovery
Date Thu, 15 Sep 2016 22:48:20 GMT

     [ https://issues.apache.org/jira/browse/YARN-5638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Li Lu updated YARN-5638:
------------------------
    Attachment: YARN-5638-trunk.v1.patch

First version of patch to demonstrate the idea. Right now I've split collector discovery process
into two steps: In the first step, collector manager reports the collector to NM, and NM sends
the collector data to the RM for registration. In the second step, the RM (synchronously)
assigns the collector a timestamp (rm's timestamp and a version number) and store it in memory.
The RM then updates known collector data via heartbeats to all NMs as before. The only difference
is the RM attach timestamp information for each collector to NMs so that once there's a rebuild
process, NMs can report this information. 

> Introduce a collector timestamp to uniquely identify collectors creation order in collector
discovery
> -----------------------------------------------------------------------------------------------------
>
>                 Key: YARN-5638
>                 URL: https://issues.apache.org/jira/browse/YARN-5638
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: YARN-5638-trunk.v1.patch
>
>
> As discussed in YARN-3359, we need to further identify timeline collectors' creation
order to rebuild collector discovery data in the RM. This JIRA proposes to use <rm_timestamp,
logical_version_number> to order collectors for each application in the RM. This timestamp
can then be used when a standby RM becomes active and rebuild collector discovery data. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message