hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris K Wensel <ch...@wensel.net>
Subject Re: Cascading
Date Wed, 23 Jan 2008 20:39:54 GMT
Hey thanks!

I wasn't aware of Dryad until after I got started. But it is much  
lower level than Cascading.

That is, I believe you can implement the MapReduce model using Dryad,  
Cascading (currently) relies on the MapReduce model.

Pig and Jaql also showed up on my radar after I got started. but I see  
them as complimentary, they can participate in a Flow, or be a kind of  
Flow. that's currently undecided.

But it makes perfect sense to use Cascading to accumulate and cleanup  
data, and then hand it to Pig/Jaql for the really hard stuff, then let  
Cascading put on the finishing touches.

cheers
ckw

On Jan 23, 2008, at 12:27 PM, Steve Severance wrote:

> Hi Chris,
> 	Really interesting stuff. Definitely very useful for building more
> complex apps. I am wondering if you were inspired at all by  
> Microsofts Dryad
> project which implements a vertex passed distributed processing  
> platform.
> Once again good job.
>
> Steve
>
>> -----Original Message-----
>> From: Chris K Wensel [mailto:chris@wensel.net]
>> Sent: Wednesday, January 23, 2008 1:34 PM
>> To: core-user@hadoop.apache.org
>> Subject: Re: Cascading
>>
>> Thought I would briefly mention a new overview page that attempts to
>> explain what Cascading does in a little more detail.
>>
>> It should answer some of the questions I've received.
>>
>> http://www.cascading.org/overview.html
>>
>> I'll leave you alone now.. <grin>
>>
>> cheers
>> ckw
>>
>> On Jan 21, 2008, at 7:46 PM, Chris K Wensel wrote:
>>
>>> Hey all,
>>>
>>> Some months ago I decided to poke around deeper with Hadoop, after
>>> having been a Nutch user for some time.
>>>
>>> After way to much of the aforementioned poking, I decided to give my
>>> work a name, a home, and set it free.
>>>
>>> You can find the results here:
>>> http://www.cascading.org/
>>>
>>> It seems to be useful, but that may not hold true for anyone other
>>> than me.
>>>
>>> But do please give it a look and let me know what you think, if you
>>> are so inclined to think about it. If you use it, I would love to
>>> hear about that as well.
>>>
>>> If it works for whatever you need it for, great news. If not, feel
>>> free to complain or contribute. It's hosted on google code and
>>> subsequently has a mail list and issue tracker.
>>>
>>> The lack of comprehensive documentation is slowly being remedied.
>>>
>>> regards,
>>> ckw
>>>
>>> Chris K Wensel
>>> chris@wensel.net
>>> http://chris.wensel.net/
>>>
>>>
>>>
>>
>> Chris K Wensel
>> chris@wensel.net
>> http://chris.wensel.net/
>>
>
>
>

Chris K Wensel
chris@wensel.net
http://chris.wensel.net/




Mime
View raw message