--A key innovation here is a partitioning layout algorithm that can support fast
many to many recovery similar to HDFS but still support partitioned operation
with deterministic key placement.
Thanks for your contribution.
Is here more detail info on this point?
yes... our design document:
I actually will probably write a paper on this...
The more I started down the partitioned filesystem approach in terms of mapreduce the more I realized that there were some REALLY elegant imoplementation and design issues that I did not originally appreciate ... (so I partially got lucky).
I think this approach could be generalized to work on normal map reduce jobs without much overhead.
Founder/CEO Spinn3r.comLocation: San Francisco, CA
Skype: burtonatorSkype-in: (415) 871-0687