hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Francis.Hu <francis...@reachjunction.com>
Subject 答复: Passing an object in mapper
Date Thu, 15 Aug 2013 04:13:09 GMT
hi, 

 

what I did when data need to be saved into DB :

 

//configure DB connection info.

DBConfiguration.configureDB(config,

                    HadoopJobConfigParser.getProperty(DB_DRIVER),

                    HadoopJobConfigParser.getProperty(DB_URL),

                    HadoopJobConfigParser.getProperty(DB_USER),

                    HadoopJobConfigParser.getProperty(DB_PWD));

 

//write a OutputKeyClass like below:

public class DisplayRequestRecord implements DBWritable,
WritableComparable<DisplayRequestRecord>{

    //TODO implement write() and readFiels() methods

}

 

//set the job's output

job.setOutputKeyClass(DisplayRequestRecord.class);

job.setOutputValueClass(DisplayRequestRecord.class);

DBOutputFormat.setOutput(job, tableName(),Fields());

 

//define reducer class like below:

public class DBDisplayRequestReducerNew extends Reducer<Text,
DoubleWritable, DisplayRequestRecord, DisplayRequestRecord>{

    public void reduce(Text key, Iterable<DoubleWritable> values,

            Context context) throws IOException, InterruptedException {

        DisplayRequestRecord dr = new DisplayRequestRecord();

        context.write(dr,dr);

    }

}

 

 

Hope it helps!

 

Thanks,

Francis.Hu

 

发件人: Sivaram RL [mailto:sivaramrls@gmail.com] 
发送时间: Thursday, August 15, 2013 11:26
收件人: user@hadoop.apache.org
主题: Re: Passing an object in mapper

 

Hi,

 

In the Configuration object in your driver class you can set the properties
as key value pair. This configuration object will be set in the Job Object.
The same properties can be accessed in the mapper/reducer using the Context
Object -> getConfiguration() -> get(propertyName).

 

Hope this helps.

 

Regards,

Sivaram R L

 

On Thu, Aug 15, 2013 at 8:39 AM, jamal sasha <jamalshasha@gmail.com> wrote:

Hi,

   I am initializing an object in driver code.

For sake of argument let say I want to save data to some database..

say:

Connection con = new Connection(host, db);

Now, in reducer I want to do something like

con.write(key,value) 

 

So, how do i pass this object from driver to mapper / reducer/?
Any clue? suggestions?

Thanks

 


Mime
View raw message