hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Goeke <goeke.matt...@gmail.com>
Subject Re: a few questions
Date Fri, 09 Nov 2012 16:50:48 GMT
Inline: HTH

On Fri, Nov 9, 2012 at 10:13 AM, Kartashov, Andy <Andy.Kartashov@mpac.ca>wrote:

> Guys,
> A few questions please.
> 1. When I tried to run Oozie examples I was told to copy to copy /examples
> folder into HDFS. However when I tried to run oozie job I was told that the
> source file was not found. Well, until I cd'ed into the local directory on
> Linux and re-run the job successfully.
> What was the point copying the examples into HDFS if they are started from
> local Linux FS.
> This opens up a similar question. When I create my jobs I run .jar file
> from local FS and not from HDFS.  I have though that was the way. Can you
> actually tell hadoop where to run the MR job from: local or HDFS?
You can actually specify that your hdfs instance run off of file:/// OR
have your MR use the local runner and that may be what you are seeing in
this case. Normally with Oozie the workflow.xml needs to be mounted in HDFS
and the local config over CLI/HTTP request needs to point the job config to
the folder they are in.

> 2. I noticed that Linux created group "hadoop" and adds hdfs/mapreduce
> user to it after the installation. However, inside HDFS the group is called
> "supergroup". Can someone elaborate on that?
Supergroup in HDFS refers to whomever the superuser is currently in HDFS*
(normally it is the hdfs user because that is the user that starts the
daemon in certain distros). You can always go in and overwrite this using
chown and a local group that adds more clarity to your permissions
structure. If you are using CDH (which it sounds like from your
explanation) then the startup scripts use the hdfs user to start the
namenode/datanodes and mapreduce to start the jobtracker/tasktrackers.

*I can't remember if mapreduce is included in this group by default but I
don't think so

> AK47
> NOTICE: This e-mail message and any attachments are confidential, subject
> to copyright and may be privileged. Any unauthorized use, copying or
> disclosure is prohibited. If you are not the intended recipient, please
> delete and contact the sender immediately. Please consider the environment
> before printing this e-mail. AVIS : le présent courriel et toute pièce
> jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur
> et peuvent être couverts par le secret professionnel. Toute utilisation,
> copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le
> destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
> l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent
> courriel

View raw message