hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Thomas <keith.tho...@gmail.com>
Subject Re: The most newbie question ever
Date Fri, 22 Mar 2013 22:05:44 GMT
OK. I have kept battling through, guessing at the gaps in the getting
started page but the final command to run the hadoop-examples.jar has
blocked me. As far as I can tell there is no hadoop-examples.jar file in
the distribution.

At a higher level I must be doing something wrong. The path I've
*followed*makes it seem as if there are effectively zero getting
started docs. This
cannot be the case for such a significant piece of software. Please help
straighten me out.

On Fri, Mar 22, 2013 at 1:30 PM, Keith Thomas <keith.thomas@gmail.com>wrote:

> Cool. Thanks this should be more straightforward.
>
> The first config file mentioned here,
>
>
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html
>
> is mapred-site.xml. The instructions say to add some config info to this
> file. Navigating to the recommended directory I don't see this file:
>
> keithomas@ubuntu:~/Development/Software/hadoop-2.0.3-alpha/etc/hadoop$ ls
> -altotal 116
> drwxr-xr-x 2 keithomas keithomas 4096 Mar 21 15:01 .
> drwxr-xr-x 3 keithomas keithomas 4096 Mar 22 12:43 ..
> -rw-r--r-- 1 keithomas keithomas 3578 Feb  6 19:40 capacity-scheduler.xml
> -rw-r--r-- 1 keithomas keithomas 1335 Feb  6 19:40 configuration.xsl
> -rw-r--r-- 1 keithomas keithomas  233 Feb  6 19:40 container-executor.cfg
> -rw-r--r-- 1 keithomas keithomas  774 Feb  6 19:40 core-site.xml
> -rw-r--r-- 1 keithomas keithomas 3397 Feb  6 19:40 hadoop-env.sh
> -rw-r--r-- 1 keithomas keithomas 1774 Feb  6 19:40
> hadoop-metrics2.properties
> -rw-r--r-- 1 keithomas keithomas 2490 Feb  6 19:40
> hadoop-metrics.properties
> -rw-r--r-- 1 keithomas keithomas 9304 Feb  6 19:40 hadoop-policy.xml
> -rw-r--r-- 1 keithomas keithomas  775 Feb  6 19:40 hdfs-site.xml
> -rw-r--r-- 1 keithomas keithomas 1180 Feb  6 19:40 httpfs-env.sh
> -rw-r--r-- 1 keithomas keithomas 1657 Feb  6 19:40 httpfs-log4j.properties
> -rw-r--r-- 1 keithomas keithomas   21 Feb  6 19:40 httpfs-signature.secret
> -rw-r--r-- 1 keithomas keithomas  620 Feb  6 19:40 httpfs-site.xml
> -rw-r--r-- 1 keithomas keithomas 8735 Feb  6 19:40 log4j.properties
> -rw-r--r-- 1 keithomas keithomas 1383 Feb  6 19:40 mapred-env.sh
> -rw-r--r-- 1 keithomas keithomas 4113 Feb  6 19:40
> mapred-queues.xml.template
> -rw-r--r-- 1 keithomas keithomas  758 Feb  6 19:40 mapred-site.xml.template
> -rw-r--r-- 1 keithomas keithomas   10 Feb  6 19:40 slaves
> -rw-r--r-- 1 keithomas keithomas 2316 Feb  6 19:40 ssl-client.xml.example
> -rw-r--r-- 1 keithomas keithomas 2251 Feb  6 19:40 ssl-server.xml.example
> -rw-r--r-- 1 keithomas keithomas 2527 Feb  6 19:40 yarn-env.sh
> -rw-r--r-- 1 keithomas keithomas  690 Feb  6 19:40 yarn-site.xml
>
> Am I supposed to create my own copy from the template file with a similar
> name?
> -Keith
>
>
> On Thu, Mar 21, 2013 at 10:19 PM, Harsh J <harsh@cloudera.com> wrote:
>
>> You've downloaded a binary inclusive release and can just start using
>> it after configuring it. Mark the extracted path as HADOOP_PREFIX, and
>> look at $HADOOP_PREFIX/etc/hadoop/ for the config files. Run HDFS and
>> YARN via the $HADOOP_PREFIX/sbin/start-{dfs,yarn}.sh scripts, and of
>> course place $HADOOP_PREFIX/bin/ on your PATH.
>>
>> If you instead prefer installing 2.x RPMs or DEBs, head over to the
>> Apache Bigtop project at http://bigtop.apache.org to find the rpm/deb
>> scripts and perhaps also packages from other builders.
>>
>> Out of the box, for 2.0.3, you may also have to configure the
>> CapacityScheduler (since its the default now) to be able to run the
>> YARN services.
>>
>> On Fri, Mar 22, 2013 at 6:14 AM, Keith Thomas <keith.thomas@gmail.com>
>> wrote:
>> > I have not checked out the source code. I have downloaded the following
>> file
>> >
>> > hadoop-2.0.3-alpha.tar.gz         07-Feb-2013 03:48   89M
>> >
>> > unzipped it and now I'm trying to find the installation instructions.
>> >
>> > -Keith
>> >
>> >
>> > On Thu, Mar 21, 2013 at 5:41 PM, Hitesh Shah <hitesh@hortonworks.com>
>> wrote:
>> >>
>> >> Also, BUILDING.txt can be found at the top level directory of the
>> checked
>> >> out code.
>> >>
>> >> -- Hitesh
>> >>
>> >> On Mar 21, 2013, at 5:39 PM, Hitesh Shah wrote:
>> >>
>> >> > Assuming you have checked out the hadoop source code into
>> >> > /home/keithomas/hadoop-common/ , you need to run the maven command
>> in that
>> >> > directory and not from /home/keithomas/
>> >> >
>> >> > -- Hitesh
>> >> >
>> >> >
>> >> > On Mar 21, 2013, at 5:20 PM, Keith Thomas wrote:
>> >> >
>> >> >> Thank you for your help Mayank. I managed to google for BUILDING.TXT
>> >> >> and discovered I need maven3. I installed maven3.
>> >> >>
>> >> >> When I enter the command on the getting started link you sent me
>> this
>> >> >> happens,
>> >> >>
>> >> >> keithomas@ubuntu:~$ mvn clean install -DskipTests
>> >> >> [INFO] Scanning for projects...
>> >> >> [INFO]
>> >> >>
>> ------------------------------------------------------------------------
>> >> >> [INFO] BUILD FAILURE
>> >> >> [INFO]
>> >> >>
>> ------------------------------------------------------------------------
>> >> >> [INFO] Total time: 0.119s
>> >> >> [INFO] Finished at: Thu Mar 21 17:14:39 PDT 2013
>> >> >> [INFO] Final Memory: 3M/247M
>> >> >> [INFO]
>> >> >>
>> ------------------------------------------------------------------------
>> >> >> [ERROR] The goal you specified requires a project to execute but
>> there
>> >> >> is no POM in this directory (/home/keithomas). Please verify you
>> invoked
>> >> >> Maven from the correct directory. -> [Help 1]
>> >> >>
>> >> >> and when I enter the command you kindly sent this happens,
>> >> >>
>> >> >> keithomas@ubuntu:~$ mvn package -Pdist -DskipTests -Dtar
>> >> >> [INFO] Scanning for projects...
>> >> >> [INFO]
>> >> >>
>> ------------------------------------------------------------------------
>> >> >> [INFO] BUILD FAILURE
>> >> >> [INFO]
>> >> >>
>> ------------------------------------------------------------------------
>> >> >> [INFO] Total time: 0.114s
>> >> >> [INFO] Finished at: Thu Mar 21 17:15:11 PDT 2013
>> >> >> [INFO] Final Memory: 3M/247M
>> >> >> [INFO]
>> >> >>
>> ------------------------------------------------------------------------
>> >> >> [WARNING] The requested profile "dist" could not be activated
>> because
>> >> >> it does not exist.
>> >> >> [ERROR] The goal you specified requires a project to execute but
>> there
>> >> >> is no POM in this directory (/home/keithomas). Please verify you
>> invoked
>> >> >> Maven from the correct directory. -> [Help 1]
>> >> >> [ERROR]
>> >> >>
>> >> >>
>> >> >> On Thu, Mar 21, 2013 at 4:38 PM, Mayank Bansal <mayank@apache.org>
>> >> >> wrote:
>> >> >> There is a BUILDING.txt
>> >> >>
>> >> >> $ mvn package -Pdist -DskipTests -Dtar
>> >> >>
>> >> >> Thanks,
>> >> >> Mayank
>> >> >>
>> >> >> On Thu, Mar 21, 2013 at 4:35 PM, Keith Thomas <
>> keith.thomas@gmail.com>
>> >> >> wrote:
>> >> >>> Thanks for the great link but sorry it looks like this is going
to
>> be
>> >> >>> a long
>> >> >>> process.
>> >> >>>
>> >> >>> Step 1 is: mvn clean install -DskipTests
>> >> >>>
>> >> >>> First of all my OS tells me I need maven but it gives me two
>> choices,
>> >> >>> maven
>> >> >>> and maven 2.
>> >> >>>
>> >> >>> Is there any pre-req info for Hadoop? I've previously guessed
I
>> should
>> >> >>> installed open jdk 6. This time I guess maven 2. The result
of the
>> >> >>> 'clean
>> >> >>> install' is an error,
>> >> >>>
>> >> >>> [ERROR] BUILD ERROR
>> >> >>> [INFO]
>> >> >>>
>> >> >>>
>> ------------------------------------------------------------------------
>> >> >>> [INFO] Cannot execute mojo: clean. It requires a project with
an
>> >> >>> existing
>> >> >>> pom.xml, but the build is not using one.
>> >> >>>
>> >> >>> :(
>> >> >>>
>> >> >>> Looking ahead the next instruction is to CD into a folder that
does
>> >> >>> not
>> >> >>> exist on my machine,
>> >> >>>
>> >> >>> cd hadoop-mapreduce-project
>> >> >>>
>> >> >>> A little after that the instructions note I must have installed
>> >> >>> version
>> >> >>> 2.4.1 or greater. 2.4.1 of what?
>> >> >>>
>> >> >>>
>> >> >>> I feel I am missing the point somewhere :(
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> On Thu, Mar 21, 2013 at 4:12 PM, Mayank Bansal <mayank@apache.org>
>> >> >>> wrote:
>> >> >>>>
>> >> >>>> http://hadoop.apache.org/docs/r2.0.3-alpha/
>> >> >>>>
>> >> >>>> On Thu, Mar 21, 2013 at 4:00 PM, Keith Thomas
>> >> >>>> <keith.thomas@gmail.com>
>> >> >>>> wrote:
>> >> >>>>> After three years in a non-Hadoop Windows wilderness
I am lucky
>> >> >>>>> enough
>> >> >>>>> to
>> >> >>>>> return and wow I find Hadoop 2.0 .... Brilliant!
>> >> >>>>>
>> >> >>>>> I quickly navigated here:
>> >> >>>>>   http://hadoop.apache.org/index.html#Getting+Started
>> >> >>>>>
>> >> >>>>> and eagerly downloaded 2.0.3 alpha. Hmmm, now what?
I see the
>> >> >>>>> following
>> >> >>>>> sections,
>> >> >>>>>
>> >> >>>>> Getting Started
>> >> >>>>>
>> >> >>>>> To get started, begin here:
>> >> >>>>>
>> >> >>>>> Learn about Hadoop by reading the documentation.
>> >> >>>>> Download Hadoop from the release page.
>> >> >>>>> Discuss Hadoop on the mailing list.
>> >> >>>>>
>> >> >>>>>
>> >> >>>>> but I can't find installation instructions. I look
for the latest
>> >> >>>>> version of
>> >> >>>>> my Tom White collection and the latest book is a year
old and is
>> >> >>>>> unlikely to
>> >> >>>>> be current enough to be of use to someone as out of
date as me.
>> >> >>>>> Please
>> >> >>>>> help
>> >> >>>>> me to get started. So far I've only managed to download
and unzip
>> >> >>>>> 2.0.3
>> >> >>>>> and
>> >> >>>>> to install open jdk 6.
>> >> >>>>>
>> >> >>>>> Many thanks for your patience.
>> >> >>>>>
>> >> >>>
>> >> >>>
>> >> >>
>> >> >
>> >>
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
>

Mime
View raw message