Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EE86E10181 for ; Fri, 20 Dec 2013 04:18:07 +0000 (UTC) Received: (qmail 99806 invoked by uid 500); 20 Dec 2013 04:17:57 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 99677 invoked by uid 500); 20 Dec 2013 04:17:39 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 99581 invoked by uid 99); 20 Dec 2013 04:17:36 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 20 Dec 2013 04:17:36 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of unmeshabiju@gmail.com designates 209.85.220.179 as permitted sender) Received: from [209.85.220.179] (HELO mail-vc0-f179.google.com) (209.85.220.179) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 20 Dec 2013 04:17:29 +0000 Received: by mail-vc0-f179.google.com with SMTP id ie18so1174226vcb.10 for ; Thu, 19 Dec 2013 20:17:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=/NNuUZhWhr1KDioQp31MDy8v0UjKXdO1Li1SaTqQXaM=; b=v0IL0AUqGURmNFFAmLnyVT9/Ux8pBmc9jxA4l0xXujgG5O85xPAXtK2dZNRZDdqNIp CjkNNJeXuwYoZtVIo8iL4BqhdQGz4dq3xz9/TMAbIuG0cs0oLMjSXir6srJOcEvnjSea EAj5eqGiShM4jxkpdBi5XjNpaehrA5jQa+3HkLzMXGvVVYNEyTEfRZveS1XpaxZR4aCU NdVKwz8d1HOv2owegB+6mom8SvAqUqDTucX7pKaCKQc2sn2Mc5mn1e1exJ67cWxMc8hf fNtAdVo92tpIll6fkLqs4suBOd2LV85Ees5WiXaib7Y+mui0PhZ4cscmwxSnfpCsGPp9 fUFQ== MIME-Version: 1.0 X-Received: by 10.52.74.99 with SMTP id s3mr42970vdv.42.1387513028788; Thu, 19 Dec 2013 20:17:08 -0800 (PST) Received: by 10.59.8.2 with HTTP; Thu, 19 Dec 2013 20:17:08 -0800 (PST) Date: Fri, 20 Dec 2013 09:47:08 +0530 Message-ID: Subject: LOGGING in MapReduce From: unmesha sreeveni To: User Hadoop Content-Type: multipart/alternative; boundary=bcaec501629333c68e04edef8d74 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec501629333c68e04edef8d74 Content-Type: text/plain; charset=ISO-8859-1 I want to log my System.out.println() in console How to do that. I did the below code but it is not displaying any thing. I am using mapred api the old one. Did i do anything wrong? Code ------------------------------------------------------------ package tech; import java.io.BufferedWriter; import java.io.IOException; import java.io.OutputStreamWriter; import java.util.ArrayList; import java.util.Iterator; import java.util.StringTokenizer; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapred.MapReduceBase; import org.apache.hadoop.mapred.OutputCollector; import org.apache.hadoop.mapred.Reducer; import org.apache.hadoop.mapred.Reporter; import com.sun.org.apache.commons.logging.LogFactory; public class Reduce extends MapReduceBase implements Reducer { static int cnt =0; ArrayList ar = new ArrayList(); String data = null; *public static final com.sun.org.apache.commons.logging.Log LOG = LogFactory.getLog(Reduce.class);* public void reduce(Text key, Iterator values, OutputCollector output, Reporter reporter) throws IOException { *System.out.println("In reducer");* *LOG.info("In Reducer");* int sum = 0; String line = key.toString(); StringTokenizer itr = new StringTokenizer(line); while (values.hasNext()) { sum += values.next().get(); } output.collect(key, new IntWritable(sum)); String data = key+" "+sum; ar.add(data); *writeToFile(ar);* System.out.println("Wrote to file"); *if(LOG.isDebugEnabled()){* * LOG.debug("Log: In Reducer");* * }* ar.add("\n"); int index=Integer.parseInt(itr.nextToken()); String value=itr.nextToken(); String classLabel=itr.nextToken(); int count=sum; } public static void *writeToFile*(ArrayList text) throws IOException { System.out.println("In reduce write to file "); C45 id=new C45(); System.out.println("count "+cnt); Path input = new Path("C45/intermediate"+id.current_index+".txt"); Configuration conf = new Configuration(); FileSystem fs = FileSystem.get(conf); BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(fs.create(input, true))); for(String str: text) { bw.write(str); } bw.newLine(); bw.close(); } } 1. And how to log details: if my reducer data goes into function *writeToFile() *and creates the file in hdfs. -- *Thanks & Regards* Unmesha Sreeveni U.B *Junior Developer* --bcaec501629333c68e04edef8d74 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
I want to log my System.out.println() in console
How to do that= .
I did= the below code but it is not displaying any thing. I am using mapred api t= he old one.
Did i do anything wrong?

Code
-------------------------= -----------------------------------

<= /div>
packag= e tech;

<= font face=3D"verdana, sans-serif">import java.io.IOException;
<= div class=3D"gmail_default"> import java.io.OutputStreamWriter;
impo= rt java.util.ArrayList;
import java.util.Iterator;
import java= .util.StringTokenizer;

import org.apache.hadoop.conf.Configuration;=
import org.= apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;=
import org.apache.hadoop.mapred.MapRed= uceBase;
import org.= apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Redu= cer;
import org.= apache.hadoop.mapred.Reporter;
import com.sun.org.apache.commons.logging.L= ogFactory;

=
publi= c =A0class Reduce extends MapReduceBase
implements Reducer<Text, IntWri= table, Text, IntWritable> {

=
static int cnt =3D0;
ArrayList<String> ar =3D new ArrayList<String>();
<= span class=3D"" style=3D"white-space:pre"> String data =3D null;

=
public static final com.su= n.org.apache.commons.logging.Log LOG =3D LogFactory.getLog(Reduce.class);

=
public void reduce(Text key, = Iterator<IntWritable> values,
OutputCollector<Text, IntWrita= ble> output,
Repor= ter reporter) throws IOException {

=
System.out.println= ("In reducer");
LOG.info("In Reducer"= );

int sum =3D 0;=
String line =3D key.toString= ();
StringTokenizer itr =3D new String= Tokenizer(line);
while= (values.hasNext()) {
sum +=3D values.next().get();
}
output.collect(key, new IntWritabl= e(sum));
String data = =3D key+" "+sum;
ar.add(data);
writeToFile(ar);
System.out.println("Wrote to = file");
if(LOG.isDebugEnabled()){
= LOG.debug("Log= : In Reducer");
}

ar.add(&q= uot;\n");
int index=3DInteger.parseInt(itr.n= extToken());
String va= lue=3Ditr.nextToken();
String classLabel=3Ditr.nextToken(= );
int count=3Dsum;

=
}

public static void writeToFile(ArrayList<String&= gt; =A0text) throws IOException {
System.out.println("In reduce= write to file ");
C45 id=3Dnew C45();
System.out.println("count &q= uot;+cnt);

Path input =3D new P= ath("C45/intermediate"+id.current_index+".txt");=
Configuration conf =3D new Config= uration();
FileSystem= fs =3D FileSystem.get(conf);
BufferedWriter bw =3D new Buffere= dWriter(new
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 OutputStreamWriter(fs.create(in= put, true)));
for(String str: text) {
bw.write(str);
}
bw.newLine();
bw.close();
}

}
1. And how to log details: if = my reducer data goes into function writeToFile() and creates the fil= e in hdfs.

--
Thanks &a= mp; Regards

Unmesha Sreeveni U.B
Junior Developer

--bcaec501629333c68e04edef8d74--