hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William <wtheisin...@gmail.com>
Subject Config
Date Tue, 23 Nov 2010 20:58:42 GMT
We are currently modifying the configuration of our hadoop grid (250
machines).  The machines are homogeneous and the specs are

dual quad core cpu 18Gb ram 8x1tb drives

currently we have set this up  -

8 reduce slots at 800mb
8 map slots at 800mb

raised our io.sort.mb to 256mb

we see a lot of spilling on both maps and reduces and I am wondering what
other configs I should be looking into

Thanks

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message