hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Prakash <ravi...@ymail.com>
Subject Re: Reliability of timestamps in logs
Date Tue, 27 Jan 2015 22:04:18 GMT
I'm afraid I don't know what the "SLS" is. Obviously it shouldn't matter if it runs on the
same node. I don't think hadoop code ever updates the system clock. In fact it shouldn't even
be run with the perms to do so.It depends on log4j appenders whether they buffer and batch
the messages before writing to disk. I would think that the timestamp would still be the time
the messages were received (rather than the time they were flushed to disk)
I am not sure if you know but you can get beautiful state diagrams of the different agents
by running $mvn -Pvisualize and then using dot to convert the *.gv files to png . I'd found
that helped me a lot

     On Monday, January 26, 2015 11:20 PM, Fabio <anytek88@gmail.com> wrote:

  Yes I am, does it make a difference? SLS runs on a single machine, wrapping the RM and simulating
the nodes, thus it should use just the system time. 
 Or do you mean there is a chance it's updating the clock while the job is running?
 On 01/26/2015 08:00 PM, Ravi Prakash wrote:
 Are you running NTP?
       On Friday, January 23, 2015 12:42 AM, Fabio <anytek88@gmail.com> wrote:
 Hi guys,
 while analyzing SLS logs I noticed some unexpected behaviors, such as 
 resources requests sent before the AM container gets to a RUNNING state.
 For this reason I started wondering how reliable is the timestamp of the 
 log entries.
 Does log4j run on an independent thread? If yes, could it be the reason 
 why some log entries appear as misplaced? Or are they supposed to be in 
 strict execution order in any case?
 I ask this because I am validating a project and I need to be pretty 
 sure about what happens when.

View raw message