hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
Date Wed, 20 Jul 2016 18:12:20 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386339#comment-15386339
] 

Hadoop QA commented on MAPREDUCE-6737:
--------------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 25s {color} | {color:blue}
Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red}
The patch doesn't appear to include any new or modified tests. Please justify why no new tests
are needed for this patch. Also please list what manual steps were performed to verify this
patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 48s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 13s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 26s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 14s {color}
| {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 45s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s {color} |
{color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 22s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 21s {color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 12s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 23s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 12s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s {color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 49s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s {color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 48s {color} | {color:green}
hadoop-mapreduce-client-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s {color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 1s {color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12819056/MAPREDUCE-6737.patch
|
| JIRA Issue | MAPREDUCE-6737 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  findbugs
 checkstyle  |
| uname | Linux bb0e5c3f173d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 38128ba |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
|  Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6626/testReport/
|
| modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common |
| Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6626/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> HS: job history recovery fails with NumericFormatException if the job wasn't initted
properly
> ---------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6737
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobhistoryserver
>    Affects Versions: 2.7.0, 2.5.1
>            Reporter: Roman Gavryliuk
>         Attachments: MAPREDUCE-6737.patch
>
>
> The problem shows itself while recovering old apps information:
> 2016-07-18 16:08:35,031 WARN
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
> start time from job history file
> job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
> : java.lang.NumberFormatException: For input string: 
> ""
> The problem is in JobHistoryEventHandler.java class in the
> following part of code:
> //initialize the launchTime in the JobIndexInfo of MetaInfo
>       if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
>         JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
>         mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());
> Because of job was not initialized properly, the 'if' statement takes value
> 'false' and .setJobStartTime() is not called.
> In JobIndexInfo constructor, we have a default value for jobStartTime:
> this.jobStartTime = -1;
> When history server recovers any application's info, it passes all parameters
> to array of strings jobDetails:
> String[] jobDetails = fileName.split(DELIMITER);
> Please note, DELIMETER is initialized in the following way:
> static final String DELIMITER = "-";
> So, jobDetails array has 10 elements - job ID, submit time, username, job name,
> finish time, number of maps, number of reducers, job status, queue, and start
> time).
> If jobStartTime = -1, the minus symbol is considered as delimeter and the code
> will assign an empty string "" as a value for 9-th element in jobDetails array.
> In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
> NumberFormatException will appear while trying to parse empty string to long
> type.
> Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));
> The most simple fix is to change the value this.jobStartTime to 0 in
> JobIndexInfo constructor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message