hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adaryl \"Bob\" Wakefield, MBA" <adaryl.wakefi...@hotmail.com>
Subject Re: The future of MapReduce
Date Tue, 01 Jul 2014 21:01:58 GMT
It was a declarative statement designed to elicit further explanation.

If someone is brand new and trying to figure out how to eat the elephant as it were, you kind
of want to burn things down to their essentials. If MapReduce isn’t going to be part of
the ecosystem in the future, one does not want to spend hours learning how to write MapReduce
jobs.

B.

From: Marco Shaw 
Sent: Tuesday, July 01, 2014 3:50 PM
To: user 
Subject: Re: The future of MapReduce

Sorry, not sure if that's a question.

Hadoop v1=HDFS+MapReduce
Hadoop v2=HDFS+YARN (+ MapReduce part of the core, but now considered optional to "get work
done")

v2 adds a better resourcing framework.  Now you can run Storm, Spark, MapReduce, etc. on Hadoop
and mix-and-match jobs/tasks with whatever your requirements, which may actually be both batch
"stuff" and/or real-time.

Not sure if that clarifies things...  Just like you can evaluate all kinds of Apache ecosystems
products to meet your needs, MapReduce is no longer the only kid on the bock.




On Tue, Jul 1, 2014 at 3:07 PM, Adaryl "Bob" Wakefield, MBA <adaryl.wakefield@hotmail.com>
wrote:

  From your answer, it sounds like you need to be able to do both.

  From: Marco Shaw 
  Sent: Tuesday, July 01, 2014 10:24 AM
  To: user 
  Subject: Re: The future of MapReduce

  It depends...  It seems most are evolving from needing "lots of data crunched", to "lots
of data crunched right now".  Most are looking for *real-time* fraud detection or recommendations,
for example, which MapReduce is not ideal for.

  Marco



  On Tue, Jul 1, 2014 at 12:00 PM, Adaryl "Bob" Wakefield, MBA <adaryl.wakefield@hotmail.com>
wrote:

    “The Mahout community decided to move its codebase onto modern data processing systems
that offer a richer programming model and more efficient execution than Hadoop MapReduce.”

    Does this mean that learning MapReduce is a waste of time? Is Storm the future or are
both technologies necessary?

    B.


Mime
View raw message