hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naganarasimha Garla <naganarasimha...@apache.org>
Subject Re: How to print values in console while running MapReduce application
Date Sun, 08 Oct 2017 16:37:43 GMT
If it is for debugging purpose would advise to try Custom MR Counters !
Though you will not get it in console you will be get it from web ui for
running job too.

On Sun, Oct 8, 2017 at 9:24 PM, Harsh J <harsh@cloudera.com> wrote:

> Consider running your job in the local mode (set config '
> mapreduce.framework.name' to 'local'). Otherwise, rely on the log viewer
> from the (Job) History Server to check the console prints in each task
> (under the stdout or stderr sections).
>
> On Thu, 5 Oct 2017 at 05:15 Tanvir Rahman <tanvir9982000@gmail.com> wrote:
>
>> Hello Demian,
>> Thanks for the answer.
>>
>>    1. I am using Java for writing the MapReduce Application. Can you
>>    tell me how to do it in JAVA?
>>    2. In the mapper or reducer function, which command did you use to
>>    write the output? Is it going to write it in Log folder? I have multiple
>>    nodes and all of them may not be used running the job, How to find which
>>    logs are modified?
>>
>> Thanks
>> Tanvir
>>
>> On Wed, Oct 4, 2017 at 4:41 PM, Demian Kurejwowski <demianpelos@yahoo.com
>> > wrote:
>>
>>> i did the same tutorial, i think they only way  is doing it  outside
>>> hadoop.
>>> in the command line:
>>> cat folder/* | python mapper.py | sort | python reducer
>>>
>>>
>>> El Miércoles, 4 de octubre, 2017 16:20:31, Tanvir Rahman <
>>> tanvir9982000@gmail.com> escribió:
>>>
>>>
>>> Hello,
>>> I have a small cluster and I am running MapReduce WordCount application
>>> in it.
>>> I want to print some variable values in the console(where you can see
>>> the map and reduce job progress and other job information) for debugging
>>> purpose while running the MapReduce application.  What is the easiest way
>>> to do that?
>>>
>>> Thanks in advance.
>>>
>>> Tanvir
>>>
>>>
>>>
>>>
>>
>>
>> --
>> [image: photo]
>> *Mohammad Tanvir Rahman*
>> Teaching Assistant, University of Houston <http://www.uh.edu/>
>> 713 628 3571 | tanvir9982000@gmail.com | http://www2.cs.uh.edu/~tanvir/
>> <http://facebook.com/tanvir.rahman.771>
>> <http://us.linkedin.com/in/tanvir9982000>
>>
>>

Mime
View raw message