hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-7062) Update mapreduce.job.tags description for making use for ATSv2 purpose.
Date Wed, 11 Apr 2018 04:36:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-7062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433393#comment-16433393
] 

Hudson commented on MAPREDUCE-7062:
-----------------------------------

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13965 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13965/])
MAPREDUCE-7062. Update mapreduce.job.tags description for making use for (rohithsharmaks:
rev 6cc59a09e7330dc893b386d84c8f2aa86c02eace)
* (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml


> Update mapreduce.job.tags description for making use for ATSv2 purpose.
> -----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-7062
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7062
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Charan Hebri
>            Assignee: Charan Hebri
>            Priority: Major
>             Fix For: 2.10.0, 3.2.0, 3.0.3, 3.1.1
>
>         Attachments: MAPREDUCE-7062.001.patch
>
>
> When applications are submitted to YARN, tags are generated in the format 
>  TIMELINE_FLOW_NAME_TAG:\{flow_name},TIMELINE_FLOW_VERSION_TAG:\{flow_version},TIMELINE_FLOW_RUN_ID_TAG:\{flow_run_id}
> However, MR applications don't follow this format and the tags submitted via the property mapreduce.job.tags are
of the format,
> {flow_name} ,\{flow_version},\{flow_run_id}
> Due to this, YARN falls back to default values for flow name, flow version and flow run
id which in turn are used in ATSv2.
> There are 2 approaches that could be taken to make MR tags compatible with ATSv2,
> Fix in the MR code
>  -------------------------
>  Prefix any tags specified with the ones needed by the YARN Timeline Service v2. But
MR is legacy code and hence these changes could affect how users are using these tags.
> Add a note in mapred-default.xml
>  --------------------------------------------
>  Add notes in the property name, mapreduce.job.tags mentioning that for purposes of
ATSv2, prefixes need to be added to the tag names.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message