Hi agai= n, I changed the String objects with org.apache.hadoop.io.Text objects (why= is String not accepted?), and now I get another exception, so I don=B4t re= ally know if I solved something or I broke something:

&n= bsp;

java.la= ng.Exception: java.lang.NullPointerException

&= nbsp; at org.apache.hadoop.mapred.LocalJobRun= ner$Job.runTasks(LocalJobRunner.java:462)

&= nbsp; at org.apache.hadoop.mapred.LocalJobRun= ner$Job.run(LocalJobRunner.java:522)

Caused = by: java.lang.NullPointerException

&= nbsp; at org.apache.hadoop.mapred.MapTask$Map= OutputBuffer.init(MapTask.java:988)

&= nbsp; at org.apache.hadoop.mapred.MapTask.cre= ateSortingCollector(MapTask.java:391)

&= nbsp; at org.apache.hadoop.mapred.MapTask.acc= ess$100(MapTask.java:80)

&= nbsp; at org.apache.hadoop.mapred.MapTask$New= OutputCollector.<init>(MapTask.java:675)

&= nbsp; at org.apache.hadoop.mapred.MapTask.run= NewMapper(MapTask.java:747)

&= nbsp; at org.apache.hadoop.mapred.MapTask.run= (MapTask.java:340)

&= nbsp; at org.apache.hadoop.mapred.LocalJobRun= ner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)

&= nbsp; at java.util.concurrent.Executors$Runna= bleAdapter.call(Executors.java:471)

&= nbsp; at java.util.concurrent.FutureTask.run(= FutureTask.java:262)

&= nbsp; at java.util.concurrent.ThreadPoolExecu= tor.runWorker(ThreadPoolExecutor.java:1145)

&= nbsp; at java.util.concurrent.ThreadPoolExecu= tor$Worker.run(ThreadPoolExecutor.java:615)

&= nbsp; at java.lang.Thread.run(Thread.java:744= )

&n= bsp;

If I co= uld debug it in my IDE, I think I could work faster, but I have the problem= s already exposed. How am I testing now? Building a jar, copying it on the = server and running a Hadoop jar command (not very performance approach…).

&n= bsp;

Could y= ou give me a hand on this? Any Windows + IntelliJ IDEa there? Maaaany t= hanks!

&n= bsp;

Von: Blanca Hernandez [mailto:Blanca.Hern= andez@willhaben.at]
Gesendet: Mittwoch, 17. September 2014 15:27
An: user@hadoop.apache.org
Betreff: ClassCastException on running map-reduce jobs + tests o= n Windows (mongo-hadoop)

Hi!

I am getting some CCE and don= =B4t really understand why…

Here my mapper:

public class AveragePriceMapper= extends Mapper<String, BSONObject, String, BSONObject>{

    @Override

    public void = map(final String key, final BSONObject val, final Context context) throws I= OException, InterruptedException {

     &= nbsp; String id =3D “result_of_making_some_operations”;

     &= nbsp; context.write(id, val);

    }=

}

And in my configuration:

config.setMapperOutputKey(Strin= g.class);

config.setMapperOutputValue(BSO= NObject.class);

On running my generated jar on = the server, seems to work everything ok until:

14/09/17 15:20:36 INFO mapred.M= apTask: Processing split: MongoInputSplit{URI=3Dmongodb://user:pass@host:27= 017/my_db.my_collection, authURI=3Dnull, min=3D{ "_id" : { "= $oid" : "541666d8e4b07265e257a42e"}}, max=3D{ }, query=3D{ }= , sort=3D{ }, fields=3D{ }, notimeout=3Dfalse}

14/09/17 15:20:36 INFO mapred.M= apTask: Map output collector class =3D org.apache.hadoop.mapred.MapTask$Map= OutputBuffer

14/09/17 15:20:36 INFO mapred.M= apTask: (EQUATOR) 0 kvi 26214396(104857584)

14/09/17 15:20:36 INFO mapred.M= apTask: mapreduce.task.io.sort.mb: 100

14/09/17 15:20:36 INFO mapred.M= apTask: soft limit at 83886080

14/09/17 15:20:36 INFO mapred.M= apTask: bufstart =3D 0; bufvoid =3D 104857600

14/09/17 15:20:36 INFO mapred.M= apTask: kvstart =3D 26214396; length =3D 6553600

14/09/17 15:20:36 INFO mapred.L= ocalJobRunner: map task executor complete.

14/09/17 15:20:36 WARN mapred.L= ocalJobRunner: job_local1701078621_0001java.lang.Exception: java.lang.Class= CastException: class java.lang.String

     &= nbsp; at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJo= bRunner.java:462)

     &= nbsp; at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunn= er.java:522)

Caused by: java.lang.ClassCastE= xception: class java.lang.String

     &= nbsp; at java.lang.Class.asSubclass(Class.java:3126)

     &= nbsp; at org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobC= onf.java:885)

     &= nbsp; at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTas= k.java:981)

     &= nbsp; at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapT= ask.java:391)

     &= nbsp; at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:80)=

     &= nbsp; at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init= >(MapTask.java:675)

     &= nbsp; at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:7= 47)

     &= nbsp; at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)<= /o:p>

     &= nbsp; at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.= run(LocalJobRunner.java:243)

     &= nbsp; at java.util.concurrent.Executors$RunnableAdapter.call(Executor= s.java:471)

     &= nbsp; at java.util.concurrent.FutureTask.run(FutureTask.java:262)

     &= nbsp; at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool= Executor.java:1145)

     &= nbsp; at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo= lExecutor.java:615)

     &= nbsp; at java.lang.Thread.run(Thread.java:744)

Did I miss something??

Another issue I am worry about:= working on a Windows system makes everything quite complicated with Hadoop= . I have it installed and running, the same as my mongoDB database (I am us= ing the connector provided by them). Running the same main class I am using in the hadooop jar call on the serv= er (in the example before), but from my IDE, I get this exception:

PriviledgedActionException as:h= ernanbl cause:java.io.IOException: Failed to set permissions of path: \tmp\= hadoop-hernanbl\mapred\staging\hernanbl1600842219\.staging to 0700

How could I make it run?

Many thanks!!=

Best regards,

Blanca