hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yoram Arnon" <yar...@yahoo-inc.com>
Subject RE: [jira] Updated: (HADOOP-307) Many small jobs benchmark for MapReduce
Date Tue, 18 Jul 2006 17:22:12 GMT
Sync'ing the jar files, in addition to adding overhead, sometimes leads to
obscure name conflicts, because the hadoop command includes
$HADOOP_HOME/hadoop-*.jar in the classpath.

> -----Original Message-----
> From: Doug Cutting [mailto:cutting@apache.org] 
> Sent: Tuesday, July 18, 2006 1:51 AM
> To: hadoop-dev@lucene.apache.org
> Subject: Re: [jira] Updated: (HADOOP-307) Many small jobs 
> benchmark for MapReduce
> Eric Baldeschwieler wrote:
> > down the road, maybe we should change the build so that 
> examples are not 
> > distributed by default, but are instead built as standalone 
> jars.  Does 
> > this make sense?
> Examples are already in separate jar files.  The problem (if any) is 
> that we sync too much stuff to the slaves.  For simplicity, 
> we sync the 
> full tree, excluding only svn files.  We could exclude more things to 
> make the sync faster, or explicitly specify the set of things to be 
> sync'd.  We do not need to sync any example code.
> In any case, I think this code more naturally belongs in src/contrib, 
> where each module has its own build.xml, rather than src/examples.
> Doug

View raw message