hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers
Date Sun, 29 Mar 2015 23:01:53 GMT

    [ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14386001#comment-14386001

Varun Saxena commented on YARN-3051:

bq. given an id, return the entity (default; see above)
bq. given an id, return all metrics of the entity
bq. given an id, return the entire config of the entity
bq. given an id, return the entity along with metrics/configs/info
Can be supported with current set of APIs'.

bq. (optional?) given an id, return one metric or some metrics (by name) of the entity (possibly
retrieving the time series of its values)
bq. (optional?) given an id, return one of some config entries (by name) of the entity
Will require additions.

bq. (need to give some more thoughts) relational queries (e.g. given an app id, return the
app entity along with its containers)
As entity id for related entities is returned in JSON response. Can't the client use that
to launch another query(for detailed info) ?

bq. From REST API's point of view, if fieldsToRetrieve is not specified, how about we return
the minimal default view? It breaks the compatibility. Since the data model is no longer compatible
anyway, it shouldn't be so much additional pain.
Yeah that is what I meant. Return whatever we decide as the minimal view if fields are not
present in REST URL.

bq. I propose the minimal default view to contain: Entity Type, Entity ID, Created Time, Modified
Sounds good.

bq. To facilitate the java developers, we can create the client lib as we did for the putting
method. We can define the advanced to methods to get the configs/metrics, which hide the detail
of composing the related params and unwrapping the response for config/metric pojo objects.
Sounds good. 

[~zjshen], noticed that YARN-3040 got in. Wanted to know, how will app context information
be used by storage implementation ? I was initially thinking it will be passed as info inside
the entity.

Anyways, do we need it during querying from reader ? AFAIK, not.
But in FileSystem Implementation, the path where entity is stored is {{root/entities/<clusterId>/<userId>/<flowId>/<flowRunId>/<appId>/<entityType>/<entityId>.thist}}.
So atleast for FS storage, reader should know all this info. Let me know if it is required
for reader or not.

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
> Per design in YARN-2928, create backing storage read interface that can be implemented
by multiple backing storage implementations.

This message was sent by Atlassian JIRA

View raw message