hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boyu Zhang <boyuzhan...@gmail.com>
Subject the time format of the log files
Date Wed, 07 Sep 2011 21:30:41 GMT
Hi all,

I need to parse the _log/history file to get the running time for map and
reduce phases. And the log is in this format:

Task TASKID="task_201109071413_0001_m_000000" TASK_TYPE="MAP" *
START_TIME="1315430092923"* SPLITS="/default-rack/dash-0-11\.sdsc\.edu" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201109071413_0001_m_000000"
TASK_ATTEMPT_ID="attempt_201109071413_0001_m_000000_0"
START_TIME="1315430092969"
TRACKER_NAME="tracker_dash-0-11\.sdsc\.edu:localhost\.localdomain/127\.0\.0\.1:33037"
HTTP_PORT="50060" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201109071413_0001_m_000000"
TASK_ATTEMPT_ID="attempt_201109071413_0001_m_000000_0" TASK_STATUS="SUCCESS"
FINISH_TIME="1315430093854" HOSTNAME="/default-rack/dash-0-11\.sdsc\.edu"
STATE_STRING=""
COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(20)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(88)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce
Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output
records)(5)][(MAP_INPUT_RECORDS)(Map input
records)(1)][(SPILLED_RECORDS)(Spilled Records)(5)][(MAP_OUTPUT_BYTES)(Map
output bytes)(40)][(COMBINE_INPUT_RECORDS)(Combine input
records)(5)][(MAP_OUTPUT_RECORDS)(Map output records)(5)]}" .
Task TASKID="task_201109071413_0001_m_000000" TASK_TYPE="MAP"
TASK_STATUS="SUCCESS"
*FINISH_TIME="1315430095939"*COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(20)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(88)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce
Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output
records)(5)][(MAP_INPUT_RECORDS)(Map input
records)(1)][(SPILLED_RECORDS)(Spilled Records)(5)][(MAP_OUTPUT_BYTES)(Map
output bytes)(40)][(COMBINE_INPUT_RECORDS)(Combine input
records)(5)][(MAP_OUTPUT_RECORDS)(Map output records)(5)]}" .
Task TASKID="task_201109071413_0001_r_000000" TASK_TYPE="REDUCE"
START_TIME="1315430095942" SPLITS="" .

I am not sure how to read this time format (milli seconds or other? ), could
anyone give some suggestions? Thanks a lot!

Thanks,
Boyu

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message