Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5CF4AFBFD for ; Wed, 27 Mar 2013 10:38:35 +0000 (UTC) Received: (qmail 72648 invoked by uid 500); 27 Mar 2013 10:38:30 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 72436 invoked by uid 500); 27 Mar 2013 10:38:30 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 72397 invoked by uid 99); 27 Mar 2013 10:38:29 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Mar 2013 10:38:29 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of zheyi.rong@gmail.com designates 209.85.212.53 as permitted sender) Received: from [209.85.212.53] (HELO mail-vb0-f53.google.com) (209.85.212.53) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Mar 2013 10:38:25 +0000 Received: by mail-vb0-f53.google.com with SMTP id fq11so1485411vbb.12 for ; Wed, 27 Mar 2013 03:38:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:x-google-sender-delegation :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=rIRMmSRoUvg4WrkltV6PCTUTUqvqpYYhbpYQ0IUEhx4=; b=lacmRTfD88+HJkdjd3sF7KmkSytEml3oGc4JLeNLPFBUxnLq5pPXjXk295p1NJJNh8 1JvMLsG91ovltgSyUD2BKNBa66mOZEo5Fo0qMSITDso9SP6+um9RFEQK43JDIHf0Lavz 014sEk6lvRkTybVQt6RT3W7uTSy+yEPvf5JrVLdml875ECqe5fdA2wgOQ4Avmkr9wzma Vtzpv/uc7N0xgzUeP/t0yUBDx9Mbk/zrwAX4kPD7GRi/HJHyFcJqSC1M5mUyO2X81Bgt Td54TnavaTePH20fiMwz50stqpQHZ1fz1zeFnGlW9eDuoX5ovfzcbkTGR8ADjLh7mFgK n8pw== MIME-Version: 1.0 X-Received: by 10.52.23.205 with SMTP id o13mr19179070vdf.78.1364380684611; Wed, 27 Mar 2013 03:38:04 -0700 (PDT) Sender: rongzheyi@gmail.com X-Google-Sender-Delegation: rongzheyi@gmail.com Received: by 10.59.9.69 with HTTP; Wed, 27 Mar 2013 03:38:04 -0700 (PDT) In-Reply-To: <1364379708.10241.YahooMailNeo@web194703.mail.sg3.yahoo.com> References: <1364379708.10241.YahooMailNeo@web194703.mail.sg3.yahoo.com> Date: Wed, 27 Mar 2013 11:38:04 +0100 X-Google-Sender-Auth: 5njklkuP9LCDv2eOWEOmKqgSNQQ Message-ID: Subject: Re: System.out.printlin vs Counters From: zheyi rong To: user@hadoop.apache.org, Sai Sai Content-Type: multipart/alternative; boundary=20cf307ca3640b786404d8e5a21a X-Virus-Checked: Checked by ClamAV on apache.org --20cf307ca3640b786404d8e5a21a Content-Type: text/plain; charset=ISO-8859-1 Hello, Q1. Depends on your need. If you would like an overall statistics, for example, the number of the malformed records in your datasets, use counters. If you just want to know what is going on inside a mapper or reducer, use System.out.println; since mappers do not know each other, you cannot get an overall statistics of your job by using System.out.println(). The output of System.out.println() will finally appear in the tasklog. Q2. In a distributed environment, mappers do not know each other. Imagine that mapper A is running on a machine, and mapper B is running on another machine, so in mapper A, you cannot get the internal state of mapper B simply by System.out.println(). Q3. Harsh J answered it. Zheyi. 2013/3/27 Sai Sai > Q1. Is it right to assume the System.out.println statements are used only > in eclipse environment and > In a multi node cluster environment we need to use counters. > > Q2. I am slightly confused as it appears like using System.out.println > statements > we r able to get detailed info at every line of code in eclipse and > counters just give few lines and not as detailed as System.out.println > statements do so what should we do in a multi node cluster enivronment. > > Q3. Also when they say the limit of counters is 120 does that mean that in > the output if we use: > context.getCounters("TestGroup1","TestName1").increment(1); > more than 120 times it will not print it. or does it refer to 120 options > of counters in an enum that we can define. > > Any help is really appreciated. > Thanks > Sai > > > --20cf307ca3640b786404d8e5a21a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello,

Q1.=A0
Depends on your need. If you would like an overall statistics, for example,= the number of the malformed records in your datasets,
use counters. If you just want to know what is going on inside a mapper or = reducer, use System.out.println;=A0
since mappers do not know each other, you cannot get an overall statistics = of your job by using System.out.println().
The output of =A0System.out.println() will finally appear in the tasklog.

Q2.
In a distributed environment, mappers do not know each other. Imagine that = mapper A is running on a machine, and mapper B is running on another machin= e, so in mapper A, you cannot get the internal state of mapper B simply by = System.out.println().

Q3.=A0
Harsh J answered it.

Zheyi.

2013/3/27 Sai Sai &l= t;saigraph@yahoo.in<= /a>>
Q1. Is it right to assume the System.out.println statements = are used only in eclipse environment and
In a multi node cluster environment we need to use counters.

=
Q2. I am slightly = confused as it appears like using System.out.println statements
we r able to get detailed info at every line of code in eclipse and co= unters just give few lines and not as detailed as System.out.println state= ments do so what should we do in a multi node cluster enivronment.

Q3. Also = when they say the limit of counters is 120 does that mean that in the outpu= t if we use:
context.getCounters(&quo= t;TestGroup1","TestName1").increment(1);
more than 120 times it will not print it. or does it refer to 120 options o= f counters in an enum that we can define.

Any help is re= ally appreciated.
Thanks
Sai



--20cf307ca3640b786404d8e5a21a--