hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Artem Onuchin <onuchin...@gmail.com>
Subject Re: TupleWritable value in mapper Not getting cleaned up ( using CompositeInputFormat )
Date Fri, 22 Mar 2013 06:49:02 GMT
Hello!

As I remember in TupleWritable there is method hasWritable. You shuld call
it in mapper to check if Writable of input 'i' is not null for each record.
22.03.2013 10:39 пользователь "devansh kumar" <devansh_kumar@yahoo.com>
написал:

> Hi,****
> ** **
> I am trying to do an outer join on to input files.****
> Can anyone help me to find out the problem here??
> ** **
> But while joining the TupleWritable value in the mapper is not getting
> cleaned up and so is using the previous values of a different key.****
> ** **
> The code I used is : (  'plist' is containing the set of paths to be taken
> as input )****
> ** **
> jobConf.setInputFormat(CompositeInputFormat.*class*);               ****
> jobConf.set("mapred.join.expr", CompositeInputFormat.*compose*(op,
> inputFormatClass,plist.toArray(*new* Path[0])));****
> jobConf.setOutputFormat(outputFormatClass);****
> ** **
> inp1:****
> ** **
> anil1     10****
> anil2     20****
> anil3     30****
> dev1     40****
> dev2     50****
> ** **
> inp2:****
> ** **
> anil1     100****
> dev1     400****
> dev2     500****
> dev3     600****
> ** **
> ** **
> outer join output:****
> ** **
> anil1     10,100****
> anil2     20,100****
> anil3     30,100****
> dev1     40,400****
> dev2     50,500****
> dev3     50,600****
> ** **
> Actually It should be, right?****
> ** **
> anil1     10,100****
> anil2     20****
> anil3     30****
> dev1     40,400****
> dev2     50,500****
> dev3     600****
> ** **
> Regards,****
> Devansh Rusia****
>

Mime
View raw message