hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-4265) Provide new timeline plugin storage to support fine-grained entity caching
Date Fri, 13 Nov 2015 00:02:11 GMT

     [ https://issues.apache.org/jira/browse/YARN-4265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Li Lu updated YARN-4265:
------------------------
    Attachment: YARN-4265.YARN-4234.001.patch

I refactored my code and added some unit tests. The current patch only depends on YARN-4234.
I addressed comments raised by [~jlowe] in the previous round of review, with two pending
actions that I think we can address in separate JIRAs:
# Build a "fall-back" plugin to have the same behavior of YARN-3942. 
# Make caching storage pluggable. 

For the second item, right now my patch does _not_ depend on YARN-4219. So I'm setting the
v1.5 plugin storage to use the memory storage system as the "caching" storage, similar to
YARN-3942.

I'm adding the v1.5 plugin storage (EntityGroupFSTimelineStore) in a module called hadoop-yarn-server-timeline-pluginstorage.
This is slightly different to YARN-3942. We need a separate module because the new v1.5 storage
depends on yarn-client, but we don't want ats server depends on yarn-client (The v1.5 storage
is not a purely server side storage.). I'm naming it as "pluginstorage" because I'm considering
to put the leveldb caching storage into this extension as well. 

> Provide new timeline plugin storage to support fine-grained entity caching
> --------------------------------------------------------------------------
>
>                 Key: YARN-4265
>                 URL: https://issues.apache.org/jira/browse/YARN-4265
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: YARN-4265-trunk.poc_001.patch, YARN-4265.YARN-4234.001.patch
>
>
> To support the newly proposed APIs in YARN-4234, we need to create a new plugin timeline
store. The store may have similar behavior as the EntityFileTimelineStore proposed in YARN-3942,
but cache date in cache id granularity, instead of application id granularity. Let's have
this storage as a standalone one, instead of updating EntityFileTimelineStore, to keep the
existing store (EntityFileTimelineStore) stable. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message