Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.210.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <1353292543.91431.YahooMailNeo@web125404.mail.ne1.yahoo.com>
References: <1353292543.91431.YahooMailNeo@web125404.mail.ne1.yahoo.com>
From: Harsh J <harsh@cloudera.com>
Date: Tue, 20 Nov 2012 00:41:13 +0530
Message-ID: 
 <CAOcnVr3LJKQqVRH52FGFwZ2FJErRqLgzWGda3zvOUh8xB7HQqw@mail.gmail.com>
Subject: Re: java.lang.ClassCastException: org.apache.hadoop.io.LongWritable
 cannot be cast to java.lang.String
To: user@hadoop.apache.org, Utester Utester <utesterp@yahoo.com>
Content-Type: text/plain; charset=ISO-8859-1

Hi,

1. Map/Reduce in 1.x. does not know how to efficiently and
automatically serialize regular Java types such as String, Long, etc..
There is experimental (and I wouldn't recommend using it either) Java
serialization support for such types in 2.x releases if you enable the
JavaSerialization classes. Please use Writable-based native type
classes (such as Text for String, LongWritable for long, etc.).

2. The type checking is done based on configuration at runtime. What
you set as JobConf.setOutput[Key/Value]Class and
JobConf.setMapOutput[Key/Value]Class (or similar new API equivalents)
will be checked for passing instances of key and value objects during
runtime. By default, these are LongWritable (key) and Text (value) and
you see the former in your error.

On Mon, Nov 19, 2012 at 8:05 AM, Utester Utester <utesterp@yahoo.com> wrote:
> Hi,
>
> I have Hadoop 1.0.3 running on Ubuntu Linux. I am playing around with a
> simple Map-Reduce job.
>
> Details of the code
> =============
> 1. Since this exercise is mainly to learn basic Hadoop APIs and learn how to
> run these jobs, the logic in the Map job is irrelevant.
> 2. The code snippet is below with just the APIs (which I believe is causing
> a problem - PLEASE correct me if I am wrong and I will post the entire code
> snippet)
> public
> class JobOrganizer
>
> ....
> ....
> public
> static class JobOrganizerMapper extends MapReduceBase implements
> Mapper<String, String, String, String> {
>
> @Override
> public void map(String sourceKey, String sourceValue,
> OutputCollector<String, String> outputC,
> Reporter reporter) throws IOException {
> ...
> ...
> }
>
> }
>
> Exception:
>
> hduser@utester-VirtualBox:/usr/local/hadoop/bin$
> /usr/local/hadoop/bin/hadoop jar
> ~/HadoopCodeProjectsFolder/JobOrganizerMapRedProject/11182012/HadoopMapRedProject.jar
> org.u.hadoopmapred.JobOrganizer
> Warning: $HADOOP_HOME is deprecated.
> 12/11/18 16:36:22 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 12/11/18 16:36:22 INFO util.NativeCodeLoader: Loaded the native-hadoop
> library
> 12/11/18 16:36:22 WARN snappy.LoadSnappy: Snappy native library not loaded
> 12/11/18 16:36:22 INFO mapred.FileInputFormat: Total input paths to process
> : 1
> 12/11/18 16:36:22 INFO mapred.JobClient: Running job: job_201211181608_0002
> 12/11/18 16:36:23 INFO mapred.JobClient:  map 0% reduce 0%
> 12/11/18 16:36:42 INFO mapred.JobClient: Task Id :
> attempt_201211181608_0002_m_000000_0, Status : FAILED
> java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be
> cast to java.lang.String
>  at
> org.u.hadoopmapred.JobOrganizer$JobOrganizerMapper.map(JobOrganizer.java:1)
>  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
>  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
>
>
> The Mapper Interface as per the javadoc is: Interface Mapper <K1, V1, K2,
> V2> and its map function is: map(K1 key, V1 value, OutputCollector<K2, V2>
> output, Reporter). I wanted to parameterize K1, V1, K2, V2 to all be String.
> Is something wrong in the way I am thinking? Is this what is wrong? I have
> found similar questions on the internet but the asnwers did not clarify how
> I am breaking the Mapper contract (I did not have any compile errors - just
> a runtime error).
>
>
>
> Thanks


-- 
Harsh J