Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 21DAA17E12 for ; Sat, 2 May 2015 20:17:12 +0000 (UTC) Received: (qmail 95850 invoked by uid 500); 2 May 2015 20:17:07 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 95751 invoked by uid 500); 2 May 2015 20:17:06 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 95738 invoked by uid 99); 2 May 2015 20:17:06 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 May 2015 20:17:06 +0000 X-ASF-Spam-Status: No, hits=2.4 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: message received from 54.164.171.186 which is an MX secondary for user@hadoop.apache.org) Received: from [54.164.171.186] (HELO mx1-us-east.apache.org) (54.164.171.186) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 May 2015 20:17:01 +0000 Received: from mail-ig0-f171.google.com (mail-ig0-f171.google.com [209.85.213.171]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 7C2844737F for ; Sat, 2 May 2015 20:16:41 +0000 (UTC) Received: by igbyr2 with SMTP id yr2so60837120igb.0 for ; Sat, 02 May 2015 13:15:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=+ge/x7whUbIMT4gM+sPECpgGR5JkwXnO6yLSN2bZcqE=; b=RBwt4sNipQqXJpIiwIgocLPEwweu2EHB3d0HUxN0u7y+b2k/F/fUHjxkE24f5RfdYZ ourzjEAKLOVpPHbDco2GYnCbLqZZs4opDoZOazklpt0sVb9YS+FM2UbThHNvXhYkWW8G mWgShIgLJXk46c6G+FDyhbMvz95u4kqlw190e+Avr5F9Fn+mpva3dZI71VqyFwqRjbMX ljrREDV8x6Y5ouPz00Yz1cePkVrZfEZahzaBk3AgXInh0z6SFmTexGBqDcN/OmsfZxkh 2QDjABR7/DIDh+zSNvkgs58OnJLhHq+YQ8+8Fg7wNpvFnWi7BHX/HQwVubhnuuyLY9Rk Dmfw== MIME-Version: 1.0 X-Received: by 10.50.66.230 with SMTP id i6mr4903135igt.22.1430597750086; Sat, 02 May 2015 13:15:50 -0700 (PDT) Received: by 10.107.12.208 with HTTP; Sat, 2 May 2015 13:15:49 -0700 (PDT) Received: by 10.107.12.208 with HTTP; Sat, 2 May 2015 13:15:49 -0700 (PDT) In-Reply-To: References: <422C3552-2674-4958-85E5-32AFE975ED41@usc.edu> Date: Sun, 3 May 2015 01:45:49 +0530 Message-ID: Subject: Re: Json Parsing in map reduce. From: Sandeep Khurana To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=047d7bd6be22b5e3e305151efe30 X-Virus-Checked: Checked by ClamAV on apache.org --047d7bd6be22b5e3e305151efe30 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable This code won't work if the json spans more than one line in the input files. On May 3, 2015 1:41 AM, "Shambhavi Punja" wrote: > Hi Shahab, > > Thanks. That helped. > > Regards, > Shambhavi > > On Thu, Apr 30, 2015 at 10:18 AM, Shahab Yunus > wrote: > >> The reason is that the Json parsing code is in a 3rd party library which >> is not included in the default map reduce/hadoop distribution. You have= to >> add them in your classpath at *runtime*. There are multiple ways to do >> it (which also depends upon how you plan to run and package/deploy your >> code.) >> >> Check out this: >> >> https://hadoopi.wordpress.com/2014/06/05/hadoop-add-third-party-librarie= s-to-mapreduce-job/ >> >> http://blog.cloudera.com/blog/2011/01/how-to-include-third-party-librari= es-in-your-map-reduce-job/ >> >> Regards, >> Shahab >> >> On Thu, Apr 30, 2015 at 1:01 PM, Shambhavi Punja wrote: >> >>> Hi, >>> >>> I am working on an assignment on Hadoop Map reduce. I am very new to Ma= p >>> Reduce. >>> >>> The assignment has many sections but for now I am trying to parse JSON >>> data. >>> >>> The input(i.e. value) to the map function is a single record of the for= m >>> xyz, {'abc=E2=80=99:=E2=80=99pqr1=E2=80=99,'abc2=E2=80=99:'pq1, pq2= =E2=80=99}, {=E2=80=98key=E2=80=99:'value1=E2=80=99} >>> I am interested only in the getting the frequency of value1. >>> >>> Following is the map- reduce job. >>> >>> public static class Map extends MapReduceBase implements >>> Mapper { >>> private final static IntWritable one =3D new IntWritable(= 1); >>> private Text word =3D new Text(); >>> >>> >>> public void map(LongWritable key, Text value, >>> OutputCollector output, Reporter reporter) throws >>> IOException { >>> String line =3D value.toString(); >>> String[] tuple =3D line.split("(?<=3D\\}),\\s"); >>> try{ >>> JSONObject obj =3D new JSONObject(tuple[1]); >>> String id =3D obj.getString(=E2=80=9Ckey"); >>> word.set(id); >>> output.collect(word, one); >>> } >>> catch(JSONException e){ >>> e.printStackTrace(); >>> } >>> } >>> } >>> >>> >>> >>> >>> public static class Reduce extends MapReduceBase implements >>> Reducer { >>> public void reduce(Text key, Iterator >>> values, OutputCollector output, Reporter reporter) >>> throws IOException { >>> int sum =3D 0; >>> while (values.hasNext()) { >>> sum +=3D values.next().get(); >>> } >>> output.collect(key, new IntWritable(sum)); >>> } >>> } >>> >>> I successfully compiled the java code using the json and hadoop jars. >>> Created a jar. But wen I run the Hadoop command I am getting the follow= ing >>> exceptions. >>> >>> >>> 15/04/30 00:36:49 WARN util.NativeCodeLoader: Unable to load >>> native-hadoop library for your platform... using builtin-java classes w= here >>> applicable >>> 15/04/30 00:36:49 WARN mapred.JobClient: Use GenericOptionsParser for >>> parsing the arguments. Applications should implement Tool for the same. >>> 15/04/30 00:36:49 WARN snappy.LoadSnappy: Snappy native library not >>> loaded >>> 15/04/30 00:36:49 INFO mapred.FileInputFormat: Total input paths to >>> process : 1 >>> 15/04/30 00:36:49 INFO mapred.JobClient: Running job: >>> job_local1121514690_0001 >>> 15/04/30 00:36:49 INFO mapred.LocalJobRunner: Waiting for map tasks >>> 15/04/30 00:36:49 INFO mapred.LocalJobRunner: Starting task: >>> attempt_local1121514690_0001_m_000000_0 >>> 15/04/30 00:36:49 INFO mapred.Task: Using ResourceCalculatorPlugin : >>> null >>> 15/04/30 00:36:49 INFO mapred.MapTask: Processing split: >>> file:/Users/Shamvi/gumgum/jars/input/ab1.txt:0+305 >>> 15/04/30 00:36:49 INFO mapred.MapTask: numReduceTasks: 1 >>> 15/04/30 00:36:49 INFO mapred.MapTask: io.sort.mb =3D 100 >>> 15/04/30 00:36:49 INFO mapred.MapTask: data buffer =3D 79691776/9961472= 0 >>> 15/04/30 00:36:49 INFO mapred.MapTask: record buffer =3D 262144/327680 >>> 15/04/30 00:36:49 INFO mapred.LocalJobRunner: Map task executor complet= e. >>> 15/04/30 00:36:49 WARN mapred.LocalJobRunner: job_local1121514690_0001 >>> java.lang.Exception: java.lang.RuntimeException: Error in configuring >>> object >>> at >>> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:354= ) >>> Caused by: java.lang.RuntimeException: Error in configuring object >>> at >>> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:= 93) >>> at >>> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) >>> at >>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java= :117) >>> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:426) >>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366) >>> at >>> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJo= bRunner.java:223) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja= va:1142) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j= ava:617) >>> at java.lang.Thread.run(Thread.java:745) >>> Caused by: java.lang.reflect.InvocationTargetException >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja= va:62) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso= rImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:483) >>> at >>> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:= 88) >>> ... 10 more >>> Caused by: java.lang.NoClassDefFoundError: org/json/JSONException >>> at java.lang.Class.forName0(Native Method) >>> at java.lang.Class.forName(Class.java:344) >>> at >>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:= 810) >>> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855= ) >>> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:881= ) >>> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:968) >>> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) >>> ... 15 more >>> Caused by: java.lang.ClassNotFoundException: org.json.JSONException >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:372) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:360) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> ... 22 more >>> 15/04/30 00:36:50 INFO mapred.JobClient: map 0% reduce 0% >>> 15/04/30 00:36:50 INFO mapred.JobClient: Job complete: >>> job_local1121514690_0001 >>> 15/04/30 00:36:50 INFO mapred.JobClient: Counters: 0 >>> 15/04/30 00:36:50 INFO mapred.JobClient: Job Failed: NA >>> Exception in thread "main" java.io.IOException: Job failed! >>> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1357) >>> at org.myorg.Wordcount.main(Wordcount.java:64) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja= va:62) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso= rImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:483) >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:160) >>> >>> >>> PS: When I modify the same code and exclude the JSON parsing i.e. find >>> frequency of {=E2=80=98key=E2=80=99:=E2=80=99value1=E2=80=99} section o= f the example input, all works well. >>> >>> >> > --047d7bd6be22b5e3e305151efe30 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

This code won't work if the json spans more than one lin= e in the input files.

On May 3, 2015 1:41 AM, "Shambhavi Punja&qu= ot; <spunja@usc.edu> wrote:
Hi Sha= hab,

Thanks. That helped.

Regar= ds,
Shambhavi

On Thu, Apr 30, 2015 at 10:18 AM, Shahab Yunus <sh= ahab.yunus@gmail.com> wrote:
The reason is that the Json parsing code is in a 3rd par= ty library which is not included in the default =C2=A0map reduce/hadoop dis= tribution. You have to add them in your classpath at runtime. There = are multiple ways to do it (which also depends upon how you plan to run and= package/deploy your code.)

Check out this:
https://hadoopi.wordpress.com/= 2014/06/05/hadoop-add-third-party-libraries-to-mapreduce-job/
=

Regards,
Shahab

On T= hu, Apr 30, 2015 at 1:01 PM, Shambhavi Punja <spunja@usc.edu> = wrote:
Hi,

I am working on an assignment on Hadoop Map reduce= . I am very new to Map Reduce.

The assignment has = many sections but for now I am trying to parse JSON data.

The input(i.e. value) to the map function is a single record of the= form =C2=A0 =C2=A0xyz, {'abc=E2=80=99:=E2=80=99pqr1=E2=80=99,'abc2= =E2=80=99:'pq1, pq2=E2=80=99}, {=E2=80=98key=E2=80=99:'value1=E2=80= =99}
I am interested only in the getting the frequency of value1.=

Following is the map- reduce job.

<= /div>
public static<= /span> class Map extends MapReduceBase impleme= nts Mapper<LongWritable, Text, Text, IntWritable> {
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 private fi= nal static IntWritable one =3D = new IntWritable(1);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 <= /span>=C2=A0 =C2=A0 =C2=A0 private Tex= t word =3D new Text();

=C2=A0=C2= =A0 =C2=A0 =C2=A0 =C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 public void map(LongWritable key, Text va= lue, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 String line =3D value.toSt= ring();
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 S= tring[] tuple =3D line.split("(?<=3D\= \}),\\s");
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 try{
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 JSONObject obj =3D = new JSONObject(tuple[1]);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 String id =3D obj.getString(=E2= =80=9Ckey");
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 word.set(id);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 output.collect(word, one);=
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
<= div style=3D"margin:0px;font-size:11px;font-family:Menlo">=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 catch(JSONException e){
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 e.printStackTrace();
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
<= div style=3D"margin:0px;font-size:11px;font-family:Menlo">=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 }

=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0

=C2=A0=C2=A0 =C2=A0

=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 public static class Reduce = extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 public void = reduce(Text key, Iterator<IntWritable> values, OutputCollector<Tex= t, IntWritable> output, Reporter reporter) throws IOException {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 int sum =3D 0;
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 while (val= ues.hasNext()) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sum += =3D values.next().get();
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
=
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 output.collect(key, new IntWritable(sum));
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 =C2=A0 =C2=A0 <= span style=3D"white-space:pre-wrap"> =C2=A0 =C2=A0 }

I successfully compiled the java code using the json and ha= doop jars. Created a jar. But wen I run the Hadoop command I am getting the= following exceptions.


15/04/30 00:36:49 WARN uti= l.NativeCodeLoader: Unable to load native-hadoop library for your platform.= .. using builtin-java classes where applicable
15/04/30 00:36:49 WARN mapred.JobClient= : Use GenericOptionsParser for parsing the arguments. Applications should i= mplement Tool for the same.
15/04/30 00:36:49 WARN snappy.LoadSnappy: Snappy native li= brary not loaded
15/04/30 00:36:49 INFO mapred.FileInputFormat: Total input paths to p= rocess : 1
= 15/04/30 00:36:49 INFO mapred.JobClient: Running job: job_local1121514690_0= 001
15/04/3= 0 00:36:49 INFO mapred.LocalJobRunner: Waiting for map tasks
15/04/30 00:36:49 INFO ma= pred.LocalJobRunner: Starting task: attempt_local1121514690_0001_m_000000_0=
15/04/30 0= 0:36:49 INFO mapred.Task:=C2=A0 Using ResourceCalculatorPlugin : null
=
15/04/30 00:36:4= 9 INFO mapred.MapTask: Processing split: file:/Users/Shamvi/gumgum/jars/inp= ut/ab1.txt:0+305
15/04/30 00:36:49 INFO mapred.MapTask: numReduceTasks: 1
15/04/30 00:36:49 INFO = mapred.MapTask: io.sort.mb =3D 100
15/04/30 00:36:49 INFO mapred.MapTask: data buffer = =3D 79691776/99614720
15/04/30 00:36:49 INFO mapred.MapTask: record buffer =3D 262144/= 327680
15/0= 4/30 00:36:49 INFO mapred.LocalJobRunner: Map task executor complete.
=
15/04/30 00:36:4= 9 WARN mapred.LocalJobRunner: job_local1121514690_0001
java.lang.Exception: java.lang.= RuntimeException: Error in configuring object
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java= :354)
Cause= d by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(R= eflectionUtils.java:93)
at org.apache.had= oop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(Re= flectionUtils.java:117)
at org.apache.had= oop.mapred.MapTask.runOldMapper(MapTask.java:426)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.LocalJobRunner$Job$Ma= pTaskRunnable.run(LocalJobRunner.java:223)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511= )
at java.util.concurrent.FutureTask.run(= FutureTask.java:266)
at java.util.concurr= ent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run= (ThreadPoolExecutor.java:617)
at java.lan= g.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.i= nvoke0(Native Method)
at sun.reflect.Nati= veMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Del= egatingMethodAccessorImpl.java:43)
at jav= a.lang.reflect.Method.invoke(Method.java:483)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.j= ava:88)
... 10 more
Caused by: java.lang.NoClassDefF= oundError: org/json/JSONException
at java= .lang.Class.forName0(Native Method)
at ja= va.lang.Class.forName(Class.java:344)
at = org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)=
at org.apache.hadoop.conf.Configuration.= getClass(Configuration.java:855)
at org.a= pache.hadoop.conf.Configuration.getClass(Configuration.java:881)
at org.apache.hadoop.mapred.JobConf.getMapperClass(= JobConf.java:968)
at org.apache.hadoop.ma= pred.MapRunner.configure(MapRunner.java:34)
... 15 more
Caused by: java.lang.ClassNotFoundException: org.json.JSONException
at java.net.URLClassLoader$1.run(URLClassLoa= der.java:372)
at java.net.URLClassLoader$= 1.run(URLClassLoader.java:361)
at java.se= curity.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassL= oader.java:424)
at java.lang.ClassLoader.= loadClass(ClassLoader.java:357)
... 22 mo= re
15/04/30= 00:36:50 INFO mapred.JobClient:=C2=A0 map 0% reduce 0%
15/04/30 00:36:50 INFO mapred.= JobClient: Job complete: job_local1121514690_0001
15/04/30 00:36:50 INFO mapred.JobCli= ent: Counters: 0
15/04/30 00:36:50 INFO mapred.JobClient: Job Failed: NA
Exception in thread &quo= t;main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1357)
at org.myorg.Wordcount.main(Wordcount.java:64= )
at sun.reflect.NativeMethodAccessorImpl= .invoke0(Native Method)
at sun.reflect.Na= tiveMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(D= elegatingMethodAccessorImpl.java:43)
at j= ava.lang.reflect.Method.invoke(Method.java:483)
= at org.apache.hadoop.util.RunJar.main(RunJar.java:160)


PS: When I modify the same code and exclud= e the JSON parsing i.e. find frequency of {=E2=80=98key=E2=80=99:=E2=80=99v= alue1=E2=80=99} section of the example input, all works well.


--047d7bd6be22b5e3e305151efe30--