hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From patf <p...@well.com>
Subject Re: Another newbie - problem with grep example
Date Sat, 24 Dec 2011 23:56:01 GMT

Thanks Prashant.

That's was yesterday on Linux at work.  It's Sat; I'm at home; and 
trying out hadoop on Windows 7.  The hadoop Windows isntall was 
something of a pain-in-the-ass but I've got hadoop basically working 
(but not computing yet).

Neither the grep nor the pi examples works.  Here's the error I see in 
my Windows installation (Windows 7 with a new cygwin install).

> c:/PROGRA~2/Java/jdk1.6.0_25/bin/java -Xmx1000m 
> -Dhadoop.log.dir=D:\downloads\hadoop\logs -Dhadoop.log.file=hadoop.log 
> -Dhadoop.home.dir=D:\downloads\hadoop\ -Dhadoop.id.str= 
> -Dhadoop.root.logger=INFO,console 
> -Dhadoop.security.logger=INFO,console 
> -Djava.library.path=/cygdrive/d/downloads/hadoop/lib/native/Windows_7-x86-32 
> -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true 
> org.apache.hadoop.util.RunJar hadoop-mapred-examples-0.22.0.jar grep 
> input output dfs[a-z.]+
> 11/12/24 15:03:27 WARN conf.Configuration: 
> mapred.used.genericoptionsparser is deprecated. Instead, use 
> mapreduce.client.genericoptionsparser.used
> 11/12/24 15:03:27 WARN mapreduce.JobSubmitter: No job jar file set.  
> User classes may not be found. See Job or Job#setJar(String).
> 11/12/24 15:03:27 INFO input.FileInputFormat: Total input paths to 
> process : 19
> 11/12/24 15:03:28 INFO mapreduce.JobSubmitter: number of splits:19
> 11/12/24 15:03:29 INFO mapreduce.Job: Running job: job_201112241459_0003
> 11/12/24 15:03:30 INFO mapreduce.Job:  map 0% reduce 0%
> 11/12/24 15:03:33 INFO mapreduce.Job: Task Id : 
> attempt_201112241459_0003_m_000020_0, Status : FAILED
> Error initializing attempt_201112241459_0003_m_000020_0:
> org.apache.hadoop.security.AccessControlException: Permission denied: 
> user=cyg_server, access=EXECUTE, inode="system":MyId:supergroup:rwx------
>         at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:205)
>         at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:161)
>         at 
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:128)
>         at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4465)
>         at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkTraverse(FSNamesystem.java:4442)
>         at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:2117)
>         at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.getFileInfo(NameNode.java:1022)
>         at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth
> 11/12/24 15:03:33 WARN mapreduce.Job: Error reading task 
> outputhttp://MyId-PC:50060/tasklog?plaintext=true&attemptid=attempt_201112241459_0003_m_000020_0&filter=stdout
user cyg_server?  That's because I'm coming in under cygwin ssh.  Looks 
like I need either change the privs in the hadoop file system or I need 
to extend more privs to cyg_server.  Although there may be an error 
(WARN) before I get to the cyg_server permissions crash?

Pat


On 12/23/2011 5:21 PM, Prashant Kommireddi wrote:
> Seems like you do not have "/user/MyId/input/conf" on HDFS.
>
> Try this.
>
> cd $HADOOP_HOME_DIR (this should be your hadoop root dir)
> hadoop fs -put conf input/conf
>
> And then run the MR job again.
>
> -Prashant Kommireddi
>
> On Fri, Dec 23, 2011 at 3:40 PM, Pat Flaherty<patf@well.com>  wrote:
>
>> Hi,
>>
>> Installed 0.22.0 on CentOS 5.7.  I can start dfs and mapred and see their
>> processes.
>>
>> Ran the first grep example: bin/hadoop jar hadoop-*-examples.jar grep
>> input output 'dfs[a-z.]+'.  It seems the correct jar name is
>> hadoop-mapred-examples-0.22.0.jar - there are no other hadoop*examples*.jar
>> files in HADOOP_HOME.
>>
>> Didn't work.  Then found and tried pi (compute pi) - that works, so my
>> installation is to some degree of approximation good.
>>
>> Back to grep.  It fails with
>>
>>> java.io.FileNotFoundException: File does not exist: /user/MyId/input/conf
>> Found and ran bin/hadoop fs -ls.  OK these directory names are internal to
>> hadoop (I assume) because Linux has no idea of /user.
>>
>> And the directory is there - but the program is failing.
>>
>> Any suggestions; where to start; etc?
>>
>> Thanks - Pat
>>


Mime
View raw message