hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From snehil wakchaure <snehi...@gmail.com>
Subject Re: The future of MapReduce
Date Tue, 01 Jul 2014 21:43:51 GMT
Heard about Google dataflow from last week
On Jul 1, 2014 4:42 PM, "Marco Shaw" <marco.shaw@gmail.com> wrote:

> Interesting timing:
> http://java.dzone.com/articles/there-future-mapreduce
>
> Google declared last week that "MapReduce was dead" more or less, but
> there are very few that process data at Google's level.
>
> Makes me wonder what Yahoo has for a tech mix these days...
>
>
>
> On Tue, Jul 1, 2014 at 6:01 PM, Adaryl "Bob" Wakefield, MBA <
> adaryl.wakefield@hotmail.com> wrote:
>
>>   It was a declarative statement designed to elicit further explanation.
>>
>> If someone is brand new and trying to figure out how to eat the elephant
>> as it were, you kind of want to burn things down to their essentials. If
>> MapReduce isn’t going to be part of the ecosystem in the future, one does
>> not want to spend hours learning how to write MapReduce jobs.
>>
>> B.
>>
>>  *From:* Marco Shaw <marco.shaw@gmail.com>
>> *Sent:* Tuesday, July 01, 2014 3:50 PM
>> *To:* user <user@hadoop.apache.org>
>> *Subject:* Re: The future of MapReduce
>>
>>  Sorry, not sure if that's a question.
>>
>> Hadoop v1=HDFS+MapReduce
>> Hadoop v2=HDFS+YARN (+ MapReduce part of the core, but now considered
>> optional to "get work done")
>>
>> v2 adds a better resourcing framework.  Now you can run Storm, Spark,
>> MapReduce, etc. on Hadoop and mix-and-match jobs/tasks with whatever your
>> requirements, which may actually be both batch "stuff" and/or real-time.
>>
>> Not sure if that clarifies things...  Just like you can evaluate all
>> kinds of Apache ecosystems products to meet your needs, MapReduce is no
>> longer the only kid on the bock.
>>
>>
>> On Tue, Jul 1, 2014 at 3:07 PM, Adaryl "Bob" Wakefield, MBA <
>> adaryl.wakefield@hotmail.com> wrote:
>>
>>>   From your answer, it sounds like you need to be able to do both.
>>>
>>>  *From:* Marco Shaw <marco.shaw@gmail.com>
>>> *Sent:* Tuesday, July 01, 2014 10:24 AM
>>> *To:* user <user@hadoop.apache.org>
>>> *Subject:* Re: The future of MapReduce
>>>
>>>  It depends...  It seems most are evolving from needing "lots of data
>>> crunched", to "lots of data crunched right now".  Most are looking for
>>> *real-time* fraud detection or recommendations, for example, which
>>> MapReduce is not ideal for.
>>>
>>> Marco
>>>
>>>
>>> On Tue, Jul 1, 2014 at 12:00 PM, Adaryl "Bob" Wakefield, MBA <
>>> adaryl.wakefield@hotmail.com> wrote:
>>>
>>>>   “The Mahout community decided to move its codebase onto modern data
>>>> processing systems that offer a richer programming model and more efficient
>>>> execution than Hadoop MapReduce.”
>>>>
>>>> Does this mean that learning MapReduce is a waste of time? Is Storm the
>>>> future or are both technologies necessary?
>>>>
>>>> B.
>>>>
>>>
>>>
>>
>>
>
>

Mime
View raw message