hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3904) Refactor timelineservice.storage to add support to online and offline aggregation writers
Date Thu, 30 Jul 2015 01:11:05 GMT

     [ https://issues.apache.org/jira/browse/YARN-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Li Lu updated YARN-3904:
    Attachment: YARN-3904-YARN-2928.005.patch

Refreshed my patch according to [~sjlee0]'s comments. Specifically, I set up a new interface
(OfflineAggregationWriter) for aggregation writers. With this new interface I decoupled PhoenixOfflineAggregationWriter
from TimelineWriter. Having a separate offline writer interface also gives us more freedom
to design the aggregation storage interface. Now in the new writer API the type of the offline
aggregation is specified by the incoming {{OfflineAggregationInfo}}. 

I also considered to combine reader and writer interfaces into a "OfflineAggregationStorage"
interface, but it turned out that we may have some reader-only implementations (such as reading
app level aggregations from HBase). Separating offline readers and writers will give us more
freedom in this case. 

> Refactor timelineservice.storage to add support to online and offline aggregation writers
> -----------------------------------------------------------------------------------------
>                 Key: YARN-3904
>                 URL: https://issues.apache.org/jira/browse/YARN-3904
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: YARN-3904-YARN-2928.001.patch, YARN-3904-YARN-2928.002.patch, YARN-3904-YARN-2928.003.patch,
YARN-3904-YARN-2928.004.patch, YARN-3904-YARN-2928.005.patch
> After we finished the design for time-based aggregation, we can adopt our existing Phoenix
storage into the storage of the aggregated data. In this JIRA, I'm proposing to refactor writers
to add support to aggregation writers. Offline aggregation writers typically has less contextual
information. We can distinguish these writers by special naming. We can also use CollectorContexts
to model all contextual information and use it in our writer interfaces. 

This message was sent by Atlassian JIRA

View raw message