hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xuan Gong <xg...@hortonworks.com>
Subject Re: Log Aggregation
Date Fri, 13 Feb 2015 17:48:11 GMT
Hey, could you check whether the logs appear in HDFS ? If not, could you
check the NodeManager logs to find out when/whether the log aggregation
for that applications started ?


Xuan Gong

On 2/13/15, 6:54 AM, "Dmitry Sivachenko" <trtrmitya@gmail.com> wrote:

>I am using hadoop-2.4.1 in distributed mode.  After a job completes, logs
>are aggregated to hdfs and are available via history server.
>Sometimes logs appear very fast after the job completes (or fails), but
>sometimes it takes long (10-20-30 minutes).
>During that period history server reports:
>Logs not available for attempt_1422914757889_1881_m_000000_0. Aggregation
>may not be complete, Check back later or try the nodemanager at <host>
>It seems that it does not depend on log size, it is not so big to take 20
>minutes to copy to hdfs.
>Why this can happen?  How can I debug the issue to understand what is
>happening during that period before logs appear at history server?

View raw message