hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Paulson <epaul...@cs.wisc.edu>
Subject Re: Map performance with custom binary format
Date Tue, 28 Jul 2009 20:59:07 GMT
On Tue, Jul 28, 2009 at 01:25:49PM -0700, Ted Dunning wrote:
> On Tue, Jul 28, 2009 at 12:15 PM, william kinney
> <william.kinney@gmail.com>wrote:
> 
> >
> > Also, from the job page (different job, same Map method, just more
> > data...~40GB. 781 files):
> > Map input records       629,738,080
> > Map input bytes         41,538,992,880
> >
> > Anything else I can look into?
> 
> 
> Yes.  The number of data local maps and how many maps total.
> 

Do "data local maps" short-circuit to the local filesystem at all, or do
they read data over HTTP from the data node's jetty instance over the
loopback device?

-Erik

Mime
View raw message