hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-7042) Killed mapreduce job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled
Date Thu, 12 Apr 2018 11:33:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16435387#comment-16435387
] 

Rohith Sharma K S commented on MAPREDUCE-7042:
----------------------------------------------

Thanks [~xgong] for the patch. Definitely I agree that we should dispatch it separate thread
to minimize the impact. 
 I see that patch is creating a new dispatcher. How about using AsyncDispatcher in JHS and
registering JobHistoryEvent type. With this, we can have simple event handler code as current
patch is doing. Using AsyncDispatcher minimize the creating of new thread with runnable implementation.
Almost very similar to RMStateStore implementation.

> Killed mapreduce job data does not move to mapreduce.jobhistory.done-dir when ATS v2
is enabled
> -----------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-7042
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7042
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Yesha Vora
>            Assignee: Xuan Gong
>            Priority: Major
>         Attachments: MAPREDUCE-7042.1.patch
>
>
> Steps:
> 1) Start a mapreduce job
> {code}
> hadoop jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient-3.0.0.3.0.0.0-751-tests.jar
sleep "-Dmapreduce.job.user.name=hrt_qa"   -m 10 -r 1 -mt 1000  -rt 1000{code}
> 2) kill job
> 3) Validate job is present at mapreduce.jobhistory.done-dir
> mapreduce.jobhistory.done-dir dir does not have job_1516776025831_0048 entry.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message