hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3904) Refactor timelineservice.storage to add support to online and offline aggregation writers
Date Mon, 27 Jul 2015 23:43:04 GMT

     [ https://issues.apache.org/jira/browse/YARN-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Li Lu updated YARN-3904:
    Attachment: YARN-3904-YARN-2928.004.patch

Update the 004 version of the patch. This patch addresses the following two major issues:
# Rebuild the current Phoenix writer into an offline aggregation writer. Specifically, the
writer writes info and metric data into the newly created Phoenix offline aggregation table.

# Simplify writer interface by using TimelineCollectorContext. In this way both normal writers
and offline aggregation writers can use the same interface to write data. 

One thing pending discussion is about the {{aggregation}} method. I feel this method is a
little bit outdated. Could anyone remind me the assumed use case for it? Will it fit for real-time
aggregations only? 

> Refactor timelineservice.storage to add support to online and offline aggregation writers
> -----------------------------------------------------------------------------------------
>                 Key: YARN-3904
>                 URL: https://issues.apache.org/jira/browse/YARN-3904
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: YARN-3904-YARN-2928.001.patch, YARN-3904-YARN-2928.002.patch, YARN-3904-YARN-2928.003.patch,
> After we finished the design for time-based aggregation, we can adopt our existing Phoenix
storage into the storage of the aggregated data. In this JIRA, I'm proposing to refactor writers
to add support to aggregation writers. Offline aggregation writers typically has less contextual
information. We can distinguish these writers by special naming. We can also use CollectorContexts
to model all contextual information and use it in our writer interfaces. 

This message was sent by Atlassian JIRA

View raw message