hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6688) Store job configurations in Timeline Service v2
Date Sun, 01 May 2016 01:17:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265565#comment-15265565
] 

Naganarasimha G R commented on MAPREDUCE-6688:
----------------------------------------------

Thanks [~varun_saxena] for trying out the solution discussed in the patch.
But as i stated earlier, concerns configs is not handled completely, though you are putting
each entity of the list returned by the event as async, async entities are merged together
in timeline client, So ideal approach would be to do similar as done for {{EventType.JOB_FINISHED}},
if its {{EventType.JOB_SUBMITTED}} then put the entities as {{sync}} so that each entity is
ensured to go as single REST call.

HistoryEvent 
l. 45 : I would suggest to go for array itself so that we can directly pass it to  {{timelineclient.putEntities/putEntitiesAsync}}
as both accept variable args

JobHistoryEventHandler
l. 1048-1060 : As mentioned in prev lines, here we can check based on the eventType and if
required loop through the array and call  {{timelineclient.putEntities}} for each entity,
ex for JOB_SUBMITTED 
*or* just call  {{timelineclient.putEntities}} passing the entities array for EventType.JOB_FINISHED

*or* just call  {{timelineclient.putEntitiesAsyncs}}  passing the entities array for all other
type of events.

This solution looks fine as of now but would require changes if we adopt different approach
for publishing metrics and configurations as per YARN-3401.


> Store job configurations in Timeline Service v2
> -----------------------------------------------
>
>                 Key: MAPREDUCE-6688
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6688
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: applicationmaster
>            Reporter: Junping Du
>            Assignee: Varun Saxena
>              Labels: yarn-2928-1st-milestone
>         Attachments: MAPREDUCE-6688-YARN-2928.01.patch, YARN-3959-YARN-2928.01.patch
>
>
> We already have configuration field in HBase schema for application entity. We need to
make sure AM write it out when it get launched.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message