hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuan Gong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-7042) Killed mapreduce job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled
Date Wed, 11 Apr 2018 22:24:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434657#comment-16434657
] 

Xuan Gong commented on MAPREDUCE-7042:
--------------------------------------

The goal is that we handle the JHS event and ATS event separately. So if there is anything
wrong while we handle the ats event, it should not block the process of handling JHS event.

The solution we used here is:
 * create an additional queue for the timeline event
 * In JHSEventHandlingThread, we simply put the timeline event to this additional timeline
event queue.
 * Create a separate eventhandlingThread to handle the timeline event

 

[~rohithsharma] Could you review this, please?

 

> Killed mapreduce job data does not move to mapreduce.jobhistory.done-dir when ATS v2
is enabled
> -----------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-7042
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7042
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Yesha Vora
>            Assignee: Xuan Gong
>            Priority: Major
>         Attachments: MAPREDUCE-7042.1.patch
>
>
> Steps:
> 1) Start a mapreduce job
> {code}
> hadoop jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient-3.0.0.3.0.0.0-751-tests.jar
sleep "-Dmapreduce.job.user.name=hrt_qa"   -m 10 -r 1 -mt 1000  -rt 1000{code}
> 2) kill job
> 3) Validate job is present at mapreduce.jobhistory.done-dir
> mapreduce.jobhistory.done-dir dir does not have job_1516776025831_0048 entry.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message