hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Hadoop Annotations support
Date Mon, 04 Jul 2011 10:54:44 GMT
On 02/07/11 03:22, Raja Nagendra Kumar wrote:
> Hi,
> Is there any plans for Hadoop to support annodations specially for the api
> level configurations eliminations.. eg.
> conf.setMapperClass(MaxTemperatureMapper.class);
> conf.setCombinerClass(MaxTemperatureReducer.class);
> conf.setReducerClass(MaxTemperatureReducer.class);
> can be easily elimiated through proper class level annotations.

How? Would something be expected to run through the entire **/*.class to 
look for annotated classes and somehow determine which ones a specific 
job is using? A lot of our JARs contain all the classes for many 
actions, with the combination of map and reduce chosen from job to job. 
Reading in all the annotations hits startup time and would cause confusion.

What could be done would be a spring-style file (JSON, etc) that 
declares the Map, Reduce, Combiner, other conf options and builds the 
conf from it, but that could be a job submission interface in front of 
the classes. Something that would work well with the outstanding 
"provide a long-haul job submission API that is robust against Hadoop 
version changes" issue in hadoop-mapreduce.

View raw message