cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Burton <>
Subject Re: Peregrine: A new map reduce framework for iterative/pipelined jobs.
Date Tue, 27 Dec 2011 11:14:58 GMT
> A key innovation here is a partitioning layout algorithm that can support
>> fast
>> many to many recovery similar to HDFS but still support partitioned
>> operation
>> with deterministic key placement.
> Thanks for your contribution.
> Is here more detail info on this point?

yes... our design document:

I actually will probably write a paper on this...

The more I started down the partitioned filesystem approach in terms of
mapreduce the more I realized that there were some REALLY elegant
imoplementation and design issues that I did not originally appreciate ...
(so I partially got lucky).

I think this approach could be generalized to work on normal map reduce
jobs without much overhead.


Founder/CEO <>

Location: *San Francisco, CA*
Skype: *burtonator*

Skype-in: *(415) 871-0687*

View raw message