hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Savage <kevinpetersav...@hotmail.com>
Subject Sharing data between maps
Date Wed, 04 Apr 2012 21:00:11 GMT

I'm currently working on some simulation software than models engineering facilities. As input
we have two big chunks of data, one about the design of the site and one about the climate
the site is in. 

As we have an extensive set of climate data (about 1000 locations) we thought it would be
interesting to say "given this design, where would be best to build it". Hadoop seemed like
a good fit. 

However, what we have is one big file of design data that needs to go to all the maps and
many big files of climate data that need to go to one map each. I've not been able to work
out if there is a good way of doing this in Hadoop. 

Is there a good way of doing something like this?

View raw message