hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jothi Padmanabhan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-157) Job History log file format is not friendly for external tools.
Date Thu, 17 Sep 2009 05:21:57 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jothi Padmanabhan updated MAPREDUCE-157:
----------------------------------------

    Release Note: 
Changes the Job History file format to use JSON.  
Simplifies the Job History Parsing logic
Removes duplication of code between HistoryViewer and the JSP files
History Files are now named as JobID_user
Introduces a new cluster level configuration "mapreduce.cluster.jobhistory.maxage" for configuring
the amount of time history files are kept before getting cleaned up

> Job History log file format is not friendly for external tools.
> ---------------------------------------------------------------
>
>                 Key: MAPREDUCE-157
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-157
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>    Affects Versions: 0.20.1
>            Reporter: Owen O'Malley
>            Assignee: Jothi Padmanabhan
>             Fix For: 0.21.0
>
>         Attachments: mapred-157-10Sep.patch, mapred-157-15Sep-v1.patch, mapred-157-15Sep.patch,
mapred-157-16Sep-v1.patch, mapred-157-16Sep.patch, mapred-157-4Sep.patch, mapred-157-7Sep-v1.patch,
mapred-157-7Sep.patch, mapred-157-prelim.patch, MAPREDUCE-157-avro.patch, MAPREDUCE-157-avro.patch
>
>
> Currently, parsing the job history logs with external tools is very difficult because
of the format. The most critical problem is that newlines aren't escaped in the strings. That
makes using tools like grep, sed, and awk very tricky.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message