hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Kimball <aa...@cloudera.com>
Subject Re: Displaying Map output in MapReduce
Date Mon, 05 Jul 2010 07:34:02 GMT
If you set the number of reduce tasks to zero, the outputs of the mappers
will be sent directly to the OutputFormat. You can debug your map phase of a
job by disabling reduce and inspecting the mapper outputs, and then
re-enable the reducer after you've got the mapping part of the job running
correctly.

-- Aaron

On Fri, Jul 2, 2010 at 10:10 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> Minor correction - port is 50075:
> http://10.32.56.159:50075/logs/userlogs/
>
> On Fri, Jul 2, 2010 at 12:32 AM, Alberto Luengo Cabanillas <
> cabiwan@gmail.com> wrote:
>
> > Hi! Supposing that you´re using a virtualized environment, you should be
> > able to get to JobTracker Web Admin Interface from http://
> > <your_virtualized_
> > machine_ip>:50070/logs/userlogs. In case you´re running Hadoop from
> inside
> > a
> > cluster frontend, use this machine´s IP. Inside this directory, it will
> > exist another one called "userlogs" where you will see all map and reduce
> > tasks launch by the JobTracker and the TaskTrackers. These tasks will
> look
> > like 'job_[m,r]_0_xxxxxx' (or similar). Inside of every one of them there
> > will be three subdirectories (stdout, stdlog and stderr) which will
> contain
> > all your debugging messages.
> > Inside my MapReduce development programs I use the Java *LOG* class in a
> > sentence like this:
> >
> > private static final Log LOG =
> > LogFactory.getLog(<your_class_name>.class.getName());
> > LOG.info("***********INSIDE MY REDUCER SETUP**********");
> >
> > Besides these messages, a normal "System.out.println(...)" will also
> appear
> > in the above directories.
> >
> > Hope it helps!
> >
> > 2010/7/2 abc xyz <fabc_xyz111@yahoo.com>
> >
> > > Hi folks,
> > >
> > > I want to just display the output of my mapper on console or want to
> log
> > it
> > > in
> > > some file. System.out.println and Log4j's logger are not displaying the
> > > output.
> > > How can I see the intermediate results?
> > >
> > > Cheers
> > >
> > >
> > >
> >
> >
> >
> >
> >
> >
> > --
> > Alberto
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message