hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hemanth Yamijala <yhema...@yahoo-inc.com>
Subject Re: [HOD] Collecting MapReduce logs
Date Mon, 10 Mar 2008 07:47:06 GMT

Luca wrote:
> Hello everyone,
>     I wonder what is the meaning of hodring.log-destination-uri versus 
> hodring.log-dir. I'd like to collect MapReduce UI logs after a job has 
> been run and the only attribute seems to be hod.hadoop-ui-log-dir, in 
> the hod section.
log-destination-uri is a config option for uploading hadoop logs after 
the cluster is deallocated. log-dir is used to store logs generated by 
the HOD processes itself. If you want MapReduce UI logs, 
hadoop-ui-log-dir is what you want, as you rightly noted.

> With that attribute specified, logs are all grabbed in that directory, 
> producing a large amount of html files. Is there a way to collect 
> them, maybe as a .tar.gz, in a place somewhere related to the user?
Sorry, no, we don't have that option yet. In fact going forward, Hadoop 
might solve this problem on its own. HADOOP-2178 seems to be related to 
this, but I haven't looked at it too closely.

> Additionally, how do administrators specify variables in these values? 
> Which interpreter interprets them? For instance, variables specified 
> in a bash fashion like $USER in section hodring or ringmaster work 
> well (I guess they are interpreted by bash itself) but if specified in 
> the hod section they're not: I tried with
>     [hod]
>     hadoop-ui-log-dir=/somedir/$USER
> but any hod command fails displaying an error on that line.
We are definitely planning to build this capability, as part of the work 
we will be doing for HADOOP-2849.
> Cheers,
> Luca

View raw message