hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kunal Gupta" <ku...@techlead-india.com>
Subject Re: RE: How to create a composite value object for output from Map method
Date Fri, 24 Sep 2010 07:13:39 GMT
Can anyone please help me in unsubscribing from the mailing lists?i send mails to the unsubscribe
ids of different groups in response to&nbsp;which i get a confirmation mail, but the addresses
on which thoseconfirmation mails ask me to reply to are too weird and automatically&nbsp;get
shortened by my email client.- Kindly help.Kunal Gupta

On Sun, 19 Sep 2010 21:15:19 +0530  wrote
&gt;



I think your first approach at serializing is correct, except
for the use of ObjectWritable.&nbsp; From the docs, ObjectWritable only handles
Strings, Arrays, and primitives.&nbsp; You are trying to use it to serialize your ArrayList.&nbsp;
Try converting the ArrayList to an array of Strings first.



As for the second problem, I’d have a look at Cascading



Hope these help…



Chris







From: Bhaskar Ghosh
[mailto:bjgindia@yahoo.co.in] 

Sent: Sunday, September 19, 2010 6:22 AM

To: mapreduce-user@hadoop.apache.org

Subject: How to create a composite value object for output from Map
method













Hi All,











What would be the
right approach to solve this problem:






 I need to
     output an object as the value from my map method. The object's class
     should have two mambers: an ArrayList and another, an
     integer.

     

     I used following two ways, but they are not working:



 I
     wrote a class MyCompositeValueWritable that implements Writable
     interface.
 Inside
     the overridden readFields and write methods, I try to read/write using the
     ObjectWritable class.
 [see
     attached file MyWordCount_ObjVal1_2.java]

     

     
 
 The
     custom class is a plain class 'MyCompositeValue' not implementing or
     inheriting anything.
 The
     Map and Reduce methods try to output the 
     MyCompositeValue&gt; &gt; using the ObjectWritable class.
 [see
     attached file Case2.txt]
 
 
 Am I going wrong somewhere? Appreciate any help.





 I have
     another problem, in which I need two types of mappers and reducer, and I
     want to execute them in this order:



 
  
   Mapper1
       -&gt; Reducer1 -&gt; Mapper2 -&gt; Reducer2
   Is it
       possible through ChainMapper and/or ChainReducer classes? It yes, then
       how? Can anybody provide some starting working example, or point me to
       some good url for the same?
   Currently,
       I am doing it as a work-around:
  
 



 
  
   
    The first
        set of Mapper-Reducer write to HDFS. Then the second set of
        Mapper-Reducer pick up that output file from HDFS and writes further
        processed output to another HDFS directory.
   
  
 



 
  
   An example
       would be really really helpful.
  
 










Thanks

Bhaskar Ghosh



"Ignorance is Bliss... Knowledge never brings Peace!!!"













Mime
View raw message