hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Istabrak Abdul-Fatah <ifa...@gmail.com>
Subject Issues and Questions on running Hadoop 2.7 Yarn MapReduce examples
Date Tue, 11 Aug 2015 13:51:13 GMT
Greetings to all,
I have installed and configured Hadoop2.7.0 on a linux VM. Then I
successfully ran the pre-compiled/packaged examples (e.g. PI, WordCount
I also downloaded the Hadoop2.7.0 source code and created an eclipse
I exported the WordCount jar file and tried to run the example from the
command line as follows:

> yarn jar /opt/yarn/my_examples/WordCount.jar
/user/yarn/input/wordcount.txt output

Q1: When I used the default WordCount implementation (shown in listing1),
it failed with a list of exceptions and suggestions to use the Tool and
toolRunner interfaces/utils (see errorListing1).
      I updated the code (see listing2) and included these suggested utils
and it ran successfully.
     Could you please provide an explanation as to why the application
failed to run in the first attempt and the necessity to use the
Tool/tooRunner utils?

Q2: Does this example create a yarn client implicitly and interacts with
the Yarn layer? if not, then could you please explain how the application
interacted with the hdfs layer given that the
      Yarn later is in between?

Thx and BR


View raw message