hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-6718) add progress log to JHS during startup
Date Tue, 12 Jul 2016 22:35:20 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Haibo Chen updated MAPREDUCE-6718:
----------------------------------
    Description: 

lWhen the JHS starts up, it initializes the internal caches and storage via the HistoryFileManager.
If we have a large number of existing finished jobs then we could spent minutes in this startup
phase without logging progress:
2016-03-14 10:56:01,444 INFO org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default
file system [hdfs://hadoopcdh.itnas01.ieee.org:8020]
2016-03-14 10:56:11,455 INFO org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Initializing
Existing Jobs...
2016-03-14 12:01:36,926 INFO org.apache.hadoop.mapreduce.v2.hs.CachedHistoryStorage: CachedHistoryStorage
Init
This makes it really difficult to assess if things are working correctly (it looks hung).
We can add logs to notify users of progress.

  was:
When the JHS starts up, it initializes the internal caches and storage via the HistoryFileManager.
If we have a large number of existing finished jobs then we could spent minutes in this startup
phase without logging progress:
2016-03-14 10:56:01,444 INFO org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default
file system [hdfs://hadoopcdh.itnas01.ieee.org:8020]
2016-03-14 10:56:11,455 INFO org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Initializing
Existing Jobs...
2016-03-14 12:01:36,926 INFO org.apache.hadoop.mapreduce.v2.hs.CachedHistoryStorage: CachedHistoryStorage
Init
This makes it really difficult to assess if things are working correctly (it looks hung).
We can add logs to notify users of progress.


> add progress log to JHS during startup
> --------------------------------------
>
>                 Key: MAPREDUCE-6718
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6718
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobhistoryserver
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>            Priority: Minor
>              Labels: supportability
>         Attachments: mapreduce6718.001.patch
>
>
> lWhen the JHS starts up, it initializes the internal caches and storage via the HistoryFileManager.
If we have a large number of existing finished jobs then we could spent minutes in this startup
phase without logging progress:
> 2016-03-14 10:56:01,444 INFO org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils:
Default file system [hdfs://hadoopcdh.itnas01.ieee.org:8020]
> 2016-03-14 10:56:11,455 INFO org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Initializing
Existing Jobs...
> 2016-03-14 12:01:36,926 INFO org.apache.hadoop.mapreduce.v2.hs.CachedHistoryStorage:
CachedHistoryStorage Init
> This makes it really difficult to assess if things are working correctly (it looks hung).
We can add logs to notify users of progress.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message