hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Norbert Burger" <norbert.bur...@gmail.com>
Subject Re: Debugging / Logging in Hadoop?
Date Thu, 30 Oct 2008 22:00:20 GMT
Seems that the slides for each of the 3 Rapleaf talks are posted in the
descriptions:

The Collector - A Tool to Have Multi-Writer Appends into HDFS
http://docs.google.com/Present?docid=dgz78tv5_10gpjhnvg9

Katta - Distributed Lucene Index in Production
http://joa23.files.wordpress.com/2008/10/katta-in-production.pdf

Debugging and Tuning
http://www.scribd.com/doc/7588384/Debugging-and-Tuning-MapReduce-Applications

Norbert

On 10/30/08, Alex Loddengaard <alex@cloudera.com> wrote:
>
> I don't think so, unfortunately.  I couldn't find them.  I remember someone
> mentioning that the slides would be posted somewhere, though.
>
> Alex
>
>
> On Thu, Oct 30, 2008 at 1:16 PM, Scott Whitecross <scott@dataxu.com>
> wrote:
>
> > Is the presentation online as well?  (Hard to see some of the slides in
> the
> > video)
> >
> >
> > On Oct 30, 2008, at 1:34 PM, Alex Loddengaard wrote:
> >
> >  Arun gave a great talk about debugging and tuning at the Rapleaf event.
> >> Take a look:
> >> <http://www.vimeo.com/2085477>
> >>
> >> Alex
> >>
> >> On Thu, Oct 30, 2008 at 6:20 AM, Malcolm Matalka <
> >> mmatalka@millennialmedia.com> wrote:
> >>
> >>  I'm not sure of the correct way, but when I need to log a job I have it
> >>> print out with some unique identifier and then just do:
> >>>
> >>> for i in list of each box; do ssh $i 'grep -R PREFIX path/to/logs';
> done
> >>>
> >>>> results
> >>>>
> >>>
> >>> It works well in a pinch
> >>>
> >>> -----Original Message-----
> >>> From: Scott Whitecross [mailto:scott@dataxu.com]
> >>> Sent: Wednesday, October 29, 2008 22:14
> >>> To: core-user@hadoop.apache.org
> >>> Subject: Debugging / Logging in Hadoop?
> >>>
> >>> I'm curious to what the best method for debugging and logging in
> >>> Hadoop?  I put together a small cluster today and a simple application
> >>> to process log files.  While it worked well, I had trouble trying to
> >>> get logging information out.  Is there any way to attach a debugger,
> >>> or get log4j to write a log file?  I tried setting up a Logger in the
> >>> class I used for the map/reduce, but I had no luck.
> >>>
> >>> Thanks.
> >>>
> >>>
> >>>
> >>>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message