hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6496) Rethink on configurations published into ATSv2
Date Wed, 26 Apr 2017 17:44:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985238#comment-15985238
] 

Varun Saxena commented on YARN-6496:
------------------------------------

[~rohithsharma], these are still effectively kV-pairs. 
Can't we flatten out the JSON representation to send key-value pairs. What I mean is KVs'
in example above can be
{code}
properties.rohith.test.properties: "inside-properties"
env.NUM_SHARDS: "insde-env"
files.type: "HADOOP_XML_TEMPLATE",
files.src_file: "hdfs://yclouddev/tmp/conf/core-site.xml",
files.dest_file: "/etc/hadoop/conf/core-site.xml"
{code}

Do you have to reconstruct the JSON representation after reading it back from ATSv2 Reader?

> Rethink on configurations published into ATSv2
> ----------------------------------------------
>
>                 Key: YARN-6496
>                 URL: https://issues.apache.org/jira/browse/YARN-6496
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>
> Slider configuration model is bit different from Hadoop. Slider has configuration object
with several fields differentiations such as properties, environment and files that has more
functionality such as fileName, fileType, srcFile, destFile. 
> {code}
> "configuration" : {
> "properties": {
> "rohith.test.properties": "inside-properties"
> },
> "env": {
> "NUM_SHARDS": "insde-env"
> }
> "files" : [
> {
> "type": "HADOOP_XML_TEMPLATE",
> "src_file": "hdfs://yclouddev/tmp/conf/core-site.xml",
> "dest_file": "/etc/hadoop/conf/core-site.xml"
> }]
> {code}
> Timeline entity config is modeled as a flattened map of  String to String, that is not
flexible enough to satisfy above use-case. This need to rethink how can it be modeled.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message