hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Wiley <kwi...@keithwiley.com>
Subject Re: Good learning resources for 0.23?
Date Thu, 24 May 2012 06:20:45 GMT
Right, thanks.  I had seen some of those resources but I'll definitely go over them in greater
detail.  To be honest, I was actually a little unclear about the backward compatibility. 
On the one hand, it is clearly documented that old MR programs should work in YARN, but on
the other hand YARN seems to infer radical changes to the allocation of resources for a job
(such as the all-important alleviation of assigning slots to mappers and reducers).  That
had me thinking that there must be crucial differences in how a job is organized and submitted,
regardless of any notion of backward compatibility.

I'll go over the various documentation again.  Thanks for the references.


On May 23, 2012, at 19:27 , Arun C Murthy wrote:

> Keith,
>  Happy to help.
>  When you mean .23 API, do you mean how to write your own applications on top of YARN?
>  If so, you can start with hadoop-2 release docs:
>  http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html
>  There is also an example application (DistributedShell) you can look at for a simpler
usage of YARN apis:
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/
>  As you are probably aware, MapReduce applications itself don't need to change when you
move to using hadoop-2.
> thanks,
> Arun
> On May 23, 2012, at 3:18 PM, Keith Wiley wrote:
>> I have already preordered the third edition of Tom's book (obviously, I don't have
it yet since it won't be published until the end of the month), but aside from that, I'm looking
for good resources for learning how to program to the .23 API.  I have found several websites
and articles that discuss the philosophical differences between .20 and .23 but I'm looking
for teaching/learning resources for getting into the guts and actually programming the thing.
 I'm pretty competent at .20 so I'm not looking for starter-level hadoop stuff.  Rather, I'm
looking for transitional resources to learn the specifics of the new design.
>> Any ideas?  How are people vetted on the older versions of hadoop learning the way
of the YARN?
>> Thanks.

Keith Wiley     kwiley@keithwiley.com     keithwiley.com    music.keithwiley.com

"The easy confidence with which I know another man's religion is folly teaches
me to suspect that my own is also."
                                           --  Mark Twain

View raw message