hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bejoy.had...@gmail.com
Subject Re: Error in configuring object
Date Sat, 17 Sep 2011 18:24:41 GMT
Shreya
       Like Harsh mentioned it is really straight forward. May be a few points to keep in
mind

1. In default the separator between key and value is tab character in the output file(Text
Output Format). But in default, csv is comma delimited/separated. So you may need to change
the separator to comma in your map reduce output.

mapred.textoutputformat.separator=','

2. Mostly multiple entities combined together makes your key as well as your value. So within
the key as well as value use a common separator to separate the entities.(Make it comma if
you need a comma delimited csv) And the same separator should be used to separate  key and
value in output file as mentioned in point 1. 

NOTE: Do a output data analysis and make sure that the separator is never a part of your data.
Choose a different  separator if you already have comma in your data.

If points 1 and 2 are satisfied then just follow Harsh's reply. With no further reformating
your csv would be good for further processing. 

Hope it helps

Regards
Bejoy K S

-----Original Message-----
From: Harsh J <harsh@cloudera.com>
Date: Thu, 15 Sep 2011 15:43:08 
To: <mapreduce-user@hadoop.apache.org>
Reply-To: mapreduce-user@hadoop.apache.org
Subject: Re: Error in configuring object

Hello Shreya,

Looks like the tasks are failing when they try to get the Mapper class
(Sentiment_Analysis_Twitter_mapper.class) instantiated. Possibly something
you might be doing in its empty constructor? Hard to tell without looking at
the mapper class since that doesn't seem to be attached.

On Thu, Sep 15, 2011 at 2:37 PM, <Shreya.Pal@cognizant.com> wrote:

> Hi,****
>
> ** **
>
> Main method looks like below:****
>
> *public* *static* *void* main(String[] args) {****
>
>             ****
>
>             JobConf conf = *new*JobConf(Sentiment_Analysis_Twitter_Driver.
> *class*);****
>
>             *sLogger*.setLevel(Level.*ALL*);****
>
>             conf.setOutputKeyClass(Text.*class*);****
>
>             conf.setOutputValueClass(Text.*class*);****
>
>             ****
>
>             *try* {****
>
>                   *cacheStopWordList*(conf);****
>
>                   *cachePositiveWordList*(conf);****
>
>                   *cacheNegativeWordList*(conf);****
>
>                   *cacheSearchList*(conf);****
>
>             } *catch* (IOException e1) {****
>
>                   // *TODO* Auto-generated catch block****
>
>                   *sLogger*.error("########copying error");****
>
>                   e1.printStackTrace();****
>
>             ****
>
>             }****
>
>             ****
>
>             *sLogger*.debug("***copied All");****
>
>             // *TODO*: specify input and output DIRECTORIES (not files)***
> *
>
>             FileInputFormat.*setInputPaths*(conf,*new* Path(
> "/home/hadoop/sa/input"));****
>
>             FileOutputFormat.*setOutputPath*(conf,*new* Path(
> "/home/hadoop/sa/output"));****
>
> ** **
>
>             // *TODO*: specify a *mapper*****
>
>             conf.setMapperClass(Sentiment_Analysis_Twitter_mapper.*class*
> );****
>
>             *sLogger*.debug("***Set All");****
>
>             *try* {****
>
>                   JobClient.*runJob*(conf); //Error comes here****
>
>             } *catch* (Exception e) {****
>
>                   *sLogger*.error("########error calling run method" +
> e.getMessage());****
>
>                   e.printStackTrace();****
>
>                   ****
>
>             }****
>
>       }****
>
> ** **
>
> Iam trying to run a MR job and get the following error:****
>
> java.lang.RuntimeException: Error in configuring object****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> ****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)***
> *
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)****
>
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:259)****
>
>         at java.security.AccessController.doPrivileged(Native Method)****
>
>         at javax.security.auth.Subject.doAs(Subject.java:415)****
>
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> ****
>
>         at org.apache.hadoop.mapred.Child.main(Child.java:253)****
>
> Caused by: java.lang.reflect.InvocationTargetException****
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)****
>
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ****
>
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ****
>
>         at java.lang.reflect.Method.invoke(Method.java:601)****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ****
>
>         ... 9 more****
>
> Caused by: java.lang.RuntimeException: java.lang.InstantiationException***
> *
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> ****
>
>         at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ****
>
>         ... 14 more****
>
> Caused by: java.lang.InstantiationException****
>
>         at
> sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
> ****
>
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> ****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
> ****
>
>         ... 15 more****
>
> ** **
>
> 11/09/15 14:29:00 INFO mapred.JobClient: Task Id :
> attempt_201108242019_0003_m_000001_0, Status : FAILED****
>
> java.lang.RuntimeException: Error in configuring object****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> ****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)***
> *
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)****
>
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:259)****
>
>         at java.security.AccessController.doPrivileged(Native Method)****
>
>         at javax.security.auth.Subject.doAs(Subject.java:415)****
>
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> ****
>
>         at org.apache.hadoop.mapred.Child.main(Child.java:253)****
>
> Caused by: java.lang.reflect.InvocationTargetException****
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)****
>
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ****
>
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ****
>
>         at java.lang.reflect.Method.invoke(Method.java:601)****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ****
>
>         ... 9 more****
>
> Caused by: java.lang.RuntimeException: java.lang.InstantiationException***
> *
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> ****
>
>         at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ****
>
>         ... 14 more****
>
> Caused by: java.lang.InstantiationException****
>
>         at
> sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
> ****
>
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> ****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
> ****
>
>         ... 15 more****
>
> ** **
>
> 11/09/15 14:29:06 INFO mapred.JobClient: Task Id :
> attempt_201108242019_0003_m_000000_1, Status : FAILED****
>
> java.lang.RuntimeException: Error in configuring object****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> ****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)***
> *
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)****
>
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:259)****
>
>         at java.security.AccessController.doPrivileged(Native Method)****
>
>         at javax.security.auth.Subject.doAs(Subject.java:415)****
>
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> ****
>
>         at org.apache.hadoop.mapred.Child.main(Child.java:253)****
>
> Caused by: java.lang.reflect.InvocationTargetException****
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)****
>
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ****
>
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ****
>
>         at java.lang.reflect.Method.invoke(Method.java:601)****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ****
>
>         ... 9 more****
>
> Caused by: java.lang.RuntimeException: java.lang.InstantiationException***
> *
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> ****
>
>         at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ****
>
>         ... 14 more****
>
> Caused by: java.lang.InstantiationException****
>
>         at
> sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
> ****
>
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> ****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
> ****
>
>         ... 15 more****
>
> ** **
>
> 11/09/15 14:29:08 INFO mapred.JobClient: Task Id :
> attempt_201108242019_0003_m_000001_1, Status : FAILED****
>
> java.lang.RuntimeException: Error in configuring object****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> ****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)***
> *
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)****
>
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:259)****
>
>         at java.security.AccessController.doPrivileged(Native Method)****
>
>         at javax.security.auth.Subject.doAs(Subject.java:415)****
>
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> ****
>
>         at org.apache.hadoop.mapred.Child.main(Child.java:253)****
>
> Caused by: java.lang.reflect.InvocationTargetException****
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)****
>
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ****
>
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ****
>
>         at java.lang.reflect.Method.invoke(Method.java:601)****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ****
>
>         ... 9 more****
>
> Caused by: java.lang.RuntimeException: java.lang.InstantiationException***
> *
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> ****
>
>         at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ****
>
>         ... 14 more****
>
> Caused by: java.lang.InstantiationException****
>
>         at
> sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
> ****
>
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> ****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
> ****
>
>         ... 15 more****
>
> ** **
>
> 11/09/15 14:29:12 INFO mapred.JobClient: Task Id :
> attempt_201108242019_0003_m_000000_2, Status : FAILED****
>
> java.lang.RuntimeException: Error in configuring object****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> ****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)***
> *
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)****
>
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:259)****
>
>         at java.security.AccessController.doPrivileged(Native Method)****
>
>         at javax.security.auth.Subject.doAs(Subject.java:415)****
>
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> ****
>
>         at org.apache.hadoop.mapred.Child.main(Child.java:253)****
>
> Caused by: java.lang.reflect.InvocationTargetException****
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)****
>
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ****
>
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ****
>
>         at java.lang.reflect.Method.invoke(Method.java:601)****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ****
>
>         ... 9 more****
>
> Caused by: java.lang.RuntimeException: java.lang.InstantiationException***
> *
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> ****
>
>         at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ****
>
>         ... 14 more****
>
> Caused by: java.lang.InstantiationException****
>
>         at
> sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
> ****
>
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> ****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
> ****
>
>         ... 15 more****
>
> ** **
>
> 11/09/15 14:29:14 INFO mapred.JobClient: Task Id :
> attempt_201108242019_0003_m_000001_2, Status : FAILED****
>
> java.lang.RuntimeException: Error in configuring object****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> ****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)***
> *
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)****
>
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:259)****
>
>         at java.security.AccessController.doPrivileged(Native Method)****
>
>         at javax.security.auth.Subject.doAs(Subject.java:415)****
>
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> ****
>
>         at org.apache.hadoop.mapred.Child.main(Child.java:253)****
>
> Caused by: java.lang.reflect.InvocationTargetException****
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)****
>
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ****
>
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ****
>
>         at java.lang.reflect.Method.invoke(Method.java:601)****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ****
>
>         ... 9 more****
>
> Caused by: java.lang.RuntimeException: java.lang.InstantiationException***
> *
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> ****
>
>         at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ****
>
>         ... 14 more****
>
> Caused by: java.lang.InstantiationException****
>
>         at
> sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
> ****
>
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> ****
>
>         at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
> ****
>
>         ... 15 more****
>
> ** **
>
> 11/09/15 14:29:24 INFO mapred.JobClient: Job complete:
> job_201108242019_0003****
>
> 11/09/15 14:29:24 INFO mapred.JobClient: Counters: 7****
>
> 11/09/15 14:29:24 INFO mapred.JobClient:   Job Counters****
>
> 11/09/15 14:29:24 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=42792****
>
> 11/09/15 14:29:24 INFO mapred.JobClient:     Total time spent by all
> reduces waiting after reserving slots (ms)=0****
>
> 11/09/15 14:29:24 INFO mapred.JobClient:     Total time spent by all maps
> waiting after reserving slots (ms)=0****
>
> 11/09/15 14:29:24 INFO mapred.JobClient:     Launched map tasks=8****
>
> 11/09/15 14:29:24 INFO mapred.JobClient:     Data-local map tasks=8****
>
> 11/09/15 14:29:24 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0****
>
> 11/09/15 14:29:24 INFO mapred.JobClient:     Failed map tasks=1****
>
> 11/09/15 14:29:24 INFO mapred.JobClient: Job Failed: NA****
>
> 11/09/15 14:29:24 ERROR sa.Sentiment_Analysis_Twitter_Driver: ########error
> calling run method Job failed!****
>
> java.io.IOException: Job failed!****
>
>         at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1204)*
> ***
>
>         at
> com.cts.sa.Sentiment_Analysis_Twitter_Driver.main(Sentiment_Analysis_Twitter_Driver.java:110)
> ****
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)****
>
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ****
>
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ****
>
>         at java.lang.reflect.Method.invoke(Method.java:601)****
>
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)****
>
> [hadoop@CTSINGTOHP6 bin]$****
>
> ** **
>
> ** **
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to the
> sender and destroy all copies of the original message. Any unauthorized
> review, use, disclosure, dissemination, forwarding, printing or copying of
> this email, and/or any action taken in reliance on the contents of this
> e-mail is strictly prohibited and may be unlawful.
>



-- 
Harsh J

Mime
View raw message