hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2033) Investigate merging generic-history into the Timeline Store
Date Tue, 15 Jul 2014 09:43:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061889#comment-14061889

Zhijie Shen commented on YARN-2033:

bq.  I would think the timeline store might have to supporting storing a lot more information
than the history store. In that case, one might want to keep them separate?

IMHO, it depends on the use cases. Either generic or the application specific metrics can
be a lot, or both. The problem to keep them separately is two have two different set of interfaces,
which double our effort of maintenance and upgrade. For example, we've done data retention,
and caching for the Leveldb-based timeline store. However, generic history cannot taken advantage
of it unless we implement the similar features for the application history store again.

I don't think keeping both metrics sets into the same store will be a big trouble for each
other. With the uniformed the store interface, we can restrict the effort of improving the
store implementation, where can isolate the two metrics sets (e.g., storing in two tables).

> Investigate merging generic-history into the Timeline Store
> -----------------------------------------------------------
>                 Key: YARN-2033
>                 URL: https://issues.apache.org/jira/browse/YARN-2033
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Zhijie Shen
>         Attachments: ProposalofStoringYARNMetricsintotheTimelineStore.pdf, YARN-2033.Prototype.patch
> Having two different stores isn't amicable to generic insights on what's happening with
applications. This is to investigate porting generic-history into the Timeline Store.
> One goal is to try and retain most of the client side interfaces as close to what we
have today.

This message was sent by Atlassian JIRA

View raw message