hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Thomas <keith.tho...@gmail.com>
Subject Re: The most newbie question ever
Date Fri, 22 Mar 2013 20:30:54 GMT
Cool. Thanks this should be more straightforward.

The first config file mentioned here,


http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html

is mapred-site.xml. The instructions say to add some config info to this
file. Navigating to the recommended directory I don't see this file:

keithomas@ubuntu:~/Development/Software/hadoop-2.0.3-alpha/etc/hadoop$ ls
-altotal 116
drwxr-xr-x 2 keithomas keithomas 4096 Mar 21 15:01 .
drwxr-xr-x 3 keithomas keithomas 4096 Mar 22 12:43 ..
-rw-r--r-- 1 keithomas keithomas 3578 Feb  6 19:40 capacity-scheduler.xml
-rw-r--r-- 1 keithomas keithomas 1335 Feb  6 19:40 configuration.xsl
-rw-r--r-- 1 keithomas keithomas  233 Feb  6 19:40 container-executor.cfg
-rw-r--r-- 1 keithomas keithomas  774 Feb  6 19:40 core-site.xml
-rw-r--r-- 1 keithomas keithomas 3397 Feb  6 19:40 hadoop-env.sh
-rw-r--r-- 1 keithomas keithomas 1774 Feb  6 19:40
hadoop-metrics2.properties
-rw-r--r-- 1 keithomas keithomas 2490 Feb  6 19:40 hadoop-metrics.properties
-rw-r--r-- 1 keithomas keithomas 9304 Feb  6 19:40 hadoop-policy.xml
-rw-r--r-- 1 keithomas keithomas  775 Feb  6 19:40 hdfs-site.xml
-rw-r--r-- 1 keithomas keithomas 1180 Feb  6 19:40 httpfs-env.sh
-rw-r--r-- 1 keithomas keithomas 1657 Feb  6 19:40 httpfs-log4j.properties
-rw-r--r-- 1 keithomas keithomas   21 Feb  6 19:40 httpfs-signature.secret
-rw-r--r-- 1 keithomas keithomas  620 Feb  6 19:40 httpfs-site.xml
-rw-r--r-- 1 keithomas keithomas 8735 Feb  6 19:40 log4j.properties
-rw-r--r-- 1 keithomas keithomas 1383 Feb  6 19:40 mapred-env.sh
-rw-r--r-- 1 keithomas keithomas 4113 Feb  6 19:40
mapred-queues.xml.template
-rw-r--r-- 1 keithomas keithomas  758 Feb  6 19:40 mapred-site.xml.template
-rw-r--r-- 1 keithomas keithomas   10 Feb  6 19:40 slaves
-rw-r--r-- 1 keithomas keithomas 2316 Feb  6 19:40 ssl-client.xml.example
-rw-r--r-- 1 keithomas keithomas 2251 Feb  6 19:40 ssl-server.xml.example
-rw-r--r-- 1 keithomas keithomas 2527 Feb  6 19:40 yarn-env.sh
-rw-r--r-- 1 keithomas keithomas  690 Feb  6 19:40 yarn-site.xml

Am I supposed to create my own copy from the template file with a similar
name?
-Keith

On Thu, Mar 21, 2013 at 10:19 PM, Harsh J <harsh@cloudera.com> wrote:

> You've downloaded a binary inclusive release and can just start using
> it after configuring it. Mark the extracted path as HADOOP_PREFIX, and
> look at $HADOOP_PREFIX/etc/hadoop/ for the config files. Run HDFS and
> YARN via the $HADOOP_PREFIX/sbin/start-{dfs,yarn}.sh scripts, and of
> course place $HADOOP_PREFIX/bin/ on your PATH.
>
> If you instead prefer installing 2.x RPMs or DEBs, head over to the
> Apache Bigtop project at http://bigtop.apache.org to find the rpm/deb
> scripts and perhaps also packages from other builders.
>
> Out of the box, for 2.0.3, you may also have to configure the
> CapacityScheduler (since its the default now) to be able to run the
> YARN services.
>
> On Fri, Mar 22, 2013 at 6:14 AM, Keith Thomas <keith.thomas@gmail.com>
> wrote:
> > I have not checked out the source code. I have downloaded the following
> file
> >
> > hadoop-2.0.3-alpha.tar.gz         07-Feb-2013 03:48   89M
> >
> > unzipped it and now I'm trying to find the installation instructions.
> >
> > -Keith
> >
> >
> > On Thu, Mar 21, 2013 at 5:41 PM, Hitesh Shah <hitesh@hortonworks.com>
> wrote:
> >>
> >> Also, BUILDING.txt can be found at the top level directory of the
> checked
> >> out code.
> >>
> >> -- Hitesh
> >>
> >> On Mar 21, 2013, at 5:39 PM, Hitesh Shah wrote:
> >>
> >> > Assuming you have checked out the hadoop source code into
> >> > /home/keithomas/hadoop-common/ , you need to run the maven command in
> that
> >> > directory and not from /home/keithomas/
> >> >
> >> > -- Hitesh
> >> >
> >> >
> >> > On Mar 21, 2013, at 5:20 PM, Keith Thomas wrote:
> >> >
> >> >> Thank you for your help Mayank. I managed to google for BUILDING.TXT
> >> >> and discovered I need maven3. I installed maven3.
> >> >>
> >> >> When I enter the command on the getting started link you sent me this
> >> >> happens,
> >> >>
> >> >> keithomas@ubuntu:~$ mvn clean install -DskipTests
> >> >> [INFO] Scanning for projects...
> >> >> [INFO]
> >> >>
> ------------------------------------------------------------------------
> >> >> [INFO] BUILD FAILURE
> >> >> [INFO]
> >> >>
> ------------------------------------------------------------------------
> >> >> [INFO] Total time: 0.119s
> >> >> [INFO] Finished at: Thu Mar 21 17:14:39 PDT 2013
> >> >> [INFO] Final Memory: 3M/247M
> >> >> [INFO]
> >> >>
> ------------------------------------------------------------------------
> >> >> [ERROR] The goal you specified requires a project to execute but
> there
> >> >> is no POM in this directory (/home/keithomas). Please verify you
> invoked
> >> >> Maven from the correct directory. -> [Help 1]
> >> >>
> >> >> and when I enter the command you kindly sent this happens,
> >> >>
> >> >> keithomas@ubuntu:~$ mvn package -Pdist -DskipTests -Dtar
> >> >> [INFO] Scanning for projects...
> >> >> [INFO]
> >> >>
> ------------------------------------------------------------------------
> >> >> [INFO] BUILD FAILURE
> >> >> [INFO]
> >> >>
> ------------------------------------------------------------------------
> >> >> [INFO] Total time: 0.114s
> >> >> [INFO] Finished at: Thu Mar 21 17:15:11 PDT 2013
> >> >> [INFO] Final Memory: 3M/247M
> >> >> [INFO]
> >> >>
> ------------------------------------------------------------------------
> >> >> [WARNING] The requested profile "dist" could not be activated because
> >> >> it does not exist.
> >> >> [ERROR] The goal you specified requires a project to execute but
> there
> >> >> is no POM in this directory (/home/keithomas). Please verify you
> invoked
> >> >> Maven from the correct directory. -> [Help 1]
> >> >> [ERROR]
> >> >>
> >> >>
> >> >> On Thu, Mar 21, 2013 at 4:38 PM, Mayank Bansal <mayank@apache.org>
> >> >> wrote:
> >> >> There is a BUILDING.txt
> >> >>
> >> >> $ mvn package -Pdist -DskipTests -Dtar
> >> >>
> >> >> Thanks,
> >> >> Mayank
> >> >>
> >> >> On Thu, Mar 21, 2013 at 4:35 PM, Keith Thomas <
> keith.thomas@gmail.com>
> >> >> wrote:
> >> >>> Thanks for the great link but sorry it looks like this is going
to
> be
> >> >>> a long
> >> >>> process.
> >> >>>
> >> >>> Step 1 is: mvn clean install -DskipTests
> >> >>>
> >> >>> First of all my OS tells me I need maven but it gives me two
> choices,
> >> >>> maven
> >> >>> and maven 2.
> >> >>>
> >> >>> Is there any pre-req info for Hadoop? I've previously guessed I
> should
> >> >>> installed open jdk 6. This time I guess maven 2. The result of
the
> >> >>> 'clean
> >> >>> install' is an error,
> >> >>>
> >> >>> [ERROR] BUILD ERROR
> >> >>> [INFO]
> >> >>>
> >> >>>
> ------------------------------------------------------------------------
> >> >>> [INFO] Cannot execute mojo: clean. It requires a project with an
> >> >>> existing
> >> >>> pom.xml, but the build is not using one.
> >> >>>
> >> >>> :(
> >> >>>
> >> >>> Looking ahead the next instruction is to CD into a folder that
does
> >> >>> not
> >> >>> exist on my machine,
> >> >>>
> >> >>> cd hadoop-mapreduce-project
> >> >>>
> >> >>> A little after that the instructions note I must have installed
> >> >>> version
> >> >>> 2.4.1 or greater. 2.4.1 of what?
> >> >>>
> >> >>>
> >> >>> I feel I am missing the point somewhere :(
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Thu, Mar 21, 2013 at 4:12 PM, Mayank Bansal <mayank@apache.org>
> >> >>> wrote:
> >> >>>>
> >> >>>> http://hadoop.apache.org/docs/r2.0.3-alpha/
> >> >>>>
> >> >>>> On Thu, Mar 21, 2013 at 4:00 PM, Keith Thomas
> >> >>>> <keith.thomas@gmail.com>
> >> >>>> wrote:
> >> >>>>> After three years in a non-Hadoop Windows wilderness I
am lucky
> >> >>>>> enough
> >> >>>>> to
> >> >>>>> return and wow I find Hadoop 2.0 .... Brilliant!
> >> >>>>>
> >> >>>>> I quickly navigated here:
> >> >>>>>   http://hadoop.apache.org/index.html#Getting+Started
> >> >>>>>
> >> >>>>> and eagerly downloaded 2.0.3 alpha. Hmmm, now what? I see
the
> >> >>>>> following
> >> >>>>> sections,
> >> >>>>>
> >> >>>>> Getting Started
> >> >>>>>
> >> >>>>> To get started, begin here:
> >> >>>>>
> >> >>>>> Learn about Hadoop by reading the documentation.
> >> >>>>> Download Hadoop from the release page.
> >> >>>>> Discuss Hadoop on the mailing list.
> >> >>>>>
> >> >>>>>
> >> >>>>> but I can't find installation instructions. I look for
the latest
> >> >>>>> version of
> >> >>>>> my Tom White collection and the latest book is a year old
and is
> >> >>>>> unlikely to
> >> >>>>> be current enough to be of use to someone as out of date
as me.
> >> >>>>> Please
> >> >>>>> help
> >> >>>>> me to get started. So far I've only managed to download
and unzip
> >> >>>>> 2.0.3
> >> >>>>> and
> >> >>>>> to install open jdk 6.
> >> >>>>>
> >> >>>>> Many thanks for your patience.
> >> >>>>>
> >> >>>
> >> >>>
> >> >>
> >> >
> >>
> >
>
>
>
> --
> Harsh J
>

Mime
View raw message