hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amit Kumar Singh" <amitsi...@cse.iitb.ac.in>
Subject Sharing Memory across Map task [multiple cores] runing in same machine
Date Fri, 05 Sep 2008 08:00:09 GMT
Can we use something like RAM FS to share static data across map tasks.

1) Quadcore machine
2) 2 1-TB Disk
3) 8 GB ram,

Now Ii need ~2.7 GB ram per Map process to load some static data in memory
using which i would be processing data.(cpu intensive jobs)

Can i share memory across mappers on the same machine so that memory
footprint is less and i can run more than 4 mappers simultaneously
utilizing all 4 cores.

Can we use stuff like RamFS

View raw message