hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7272) Enable timeline collector fault tolerance
Date Mon, 09 Oct 2017 05:09:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-7272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16196504#comment-16196504
] 

Rohith Sharma K S commented on YARN-7272:
-----------------------------------------

thanks Jason for your inputs! I am looking not only RW operations but also for delete operations.
Reasons for delete is if we start keeping all the entities in level-db, it would become too
heavier like in ATS1. So only outstanding entities need to be kept under level-db which are
not flushed yet into back end. After flushing succeeded, delete from WAL which reduces size
of WAL. Advantage we get from this is recovery will be faster with only delta entities. 

bq. Leveldb is already a dependency used in multiple places, and I'd hate to see us add yet
another dependency or reinvent the wheel here.
Sorry I didn't get consensus to be taken. Shouldn't ATSv2 use level db for WAL writers?

> Enable timeline collector fault tolerance
> -----------------------------------------
>
>                 Key: YARN-7272
>                 URL: https://issues.apache.org/jira/browse/YARN-7272
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineclient, timelinereader, timelineserver
>            Reporter: Vrushali C
>            Assignee: Rohith Sharma K S
>
> If a NM goes down and along with it the timeline collector aux service for a running
yarn app, we would like that yarn app to re-establish connection with a new timeline collector.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message