hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: DataDrivenInputFormat setInput with boundingQuery
Date Thu, 02 Sep 2010 06:32:00 GMT
Thank you for mentioning this problem- it's something fairly mysterious to me.



On Tue, Aug 31, 2010 at 8:06 PM, Edward Capriolo <edlinuxguru@gmail.com> wrote:
> On Tue, Aug 31, 2010 at 10:32 PM, Edward Capriolo <edlinuxguru@gmail.com> wrote:
>> I am working with DataDrivenOutputFormat from trunk. None of the unit
>> tests seem to test the bounded queries
>>
>> Configuration conf = new Configuration();
>>                Job job = new Job(conf);
>>                job.setJarByClass(TestZ.class);
>>
>>                job.setInputFormatClass(DataDrivenDBInputFormat.class);
>>                job.setMapperClass(PrintlnMapper.class);
>>                job.setOutputFormatClass(NullOutputFormat.class);
>>                job.setMapOutputKeyClass(NullWritable.class);
>>                job.setMapOutputValueClass(NullDBWritable.class);
>>                job.setOutputKeyClass(NullWritable.class);
>>                job.setOutputValueClass(NullWritable.class);
>>
>>                job.setNumReduceTasks(0);
>>
>>                job.getConfiguration().setInt("mapreduce.map.tasks", 2);
>>
>>                DBConfiguration.configureDB(conf, "com.mysql.jdbc.Driver",
>>                                "jdbc:mysql://localhost:3306/test",
null, null);
>>
>>                DataDrivenDBInputFormat.setInput(job, NullDBWritable.class,
>>                                "SELECT * FROM name WHERE $CONDITIONS",
>>                                "SELECT MIN(id),MAX(id) FROM name");
>>                int ret = job.waitForCompletion(true) ? 0 : 1;
>>
>> Exception in thread "main" java.lang.RuntimeException:
>> java.lang.RuntimeException: java.lang.NullPointerException
>>        at org.apache.hadoop.mapreduce.lib.db.DBInputFormat.setConf(DBInputFormat.java:165)
>>
>> Can someone tell me what I am missing here?
>> Thanks,
>> Edward
>>
>
> Nevermind
>        DBConfiguration.configureDB(job.getConfiguration(), "com.mysql.jdbc.Driver",
>                                "jdbc:mysql://localhost:3306/test", null,
null);
>
> That is 4 hours of my life. I won't get back.
>



-- 
Lance Norskog
goksron@gmail.com

Mime
View raw message