Return-Path: Delivered-To: apmail-lucene-hadoop-dev-archive@locus.apache.org Received: (qmail 1172 invoked from network); 25 Mar 2006 00:04:36 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 25 Mar 2006 00:04:36 -0000 Received: (qmail 86123 invoked by uid 500); 25 Mar 2006 00:04:33 -0000 Delivered-To: apmail-lucene-hadoop-dev-archive@lucene.apache.org Received: (qmail 86038 invoked by uid 500); 25 Mar 2006 00:04:32 -0000 Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-dev@lucene.apache.org Received: (qmail 86029 invoked by uid 99); 25 Mar 2006 00:04:32 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Mar 2006 16:04:32 -0800 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [192.87.106.226] (HELO ajax.apache.org) (192.87.106.226) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Mar 2006 16:04:32 -0800 Received: from ajax (localhost.localdomain [127.0.0.1]) by ajax.apache.org (Postfix) with ESMTP id 058EDD49FC for ; Sat, 25 Mar 2006 00:04:11 +0000 (GMT) Message-ID: <2115505809.1143245051019.JavaMail.jira@ajax> Date: Sat, 25 Mar 2006 00:04:11 +0000 (GMT) From: "chris (JIRA)" To: hadoop-dev@lucene.apache.org Subject: [jira] Commented: (HADOOP-52) mapred input and output dirs must be absolute In-Reply-To: <1415168910.1140561514361.JavaMail.jira@ajax.apache.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/HADOOP-52?page=comments#action_12371823 ] chris commented on HADOOP-52: ----------------------------- I just update to the lately, start all the namenode, datanode, jobtracker and tasktracker run nutch command (crawl / readdb etc.), and jobtracker thrown exception, 060325 001911 Property 'file.separator' is \ 060325 001911 Property 'java.vendor.url.bug' is http://java.sun.com/cgi-bin/bugr eport.cgi 060325 001911 Property 'sun.io.unicode.encoding' is UnicodeLittle 060325 001911 Property 'sun.cpu.endian' is little 060325 001911 Property 'sun.desktop' is windows 060325 001911 Property 'sun.cpu.isalist' is 060325 001911 Version Jetty/5.1.4 060325 001911 Checking Resource aliases 060325 001912 Started org.mortbay.jetty.servlet.WebApplicationHandler@1d7fbfb 060325 001912 Started WebApplicationContext[/,/] 060325 001912 Started SocketListener on 0.0.0.0:50030 060325 001912 Started org.mortbay.jetty.Server@c88440 060325 001921 Server connection on port 50020 from 192.168.1.10: starting 060325 001927 Server connection on port 50020 from 192.168.1.11: starting 060325 001933 Server connection on port 50020 from 192.168.1.12: starting 060325 002240 parsing file:/D:/jobcall_trunk/hadoop/conf/hadoop-default.xml 060325 002240 parsing file:/D:/jobcall_trunk/hadoop/conf/mapred-default.xml 060325 002240 parsing file:/D:/jobcall_trunk/hadoop/conf/hadoop-site.xml 060325 002240 parsing file:/D:/jobcall_trunk/hadoop/conf/hadoop-default.xml 060325 002240 parsing file:/D:/jobcall_trunk/hadoop/conf/mapred-default.xml 060325 002240 parsing \tmp\hadoop\mapred\local\job_3dgnm3.xml\jobTracker 060325 002240 parsing file:/D:/jobcall_trunk/hadoop/conf/hadoop-site.xml 060325 002258 parsing file:/D:/jobcall_trunk/hadoop/conf/hadoop-default.xml 060325 002258 parsing file:/D:/jobcall_trunk/hadoop/conf/mapred-default.xml 060325 002258 parsing \tmp\hadoop\mapred\local\job_3dgnm3.xml\jobTracker 060325 002258 parsing file:/D:/jobcall_trunk/hadoop/conf/hadoop-site.xml 060325 002258 job init failed java.io.IOException: No input directories specified in: Configuration: defaults: hadoop-default.xml , mapred-default.xml , \tmp\hadoop\mapred\local\job_3dgnm3.x ml\jobTrackerfinal: hadoop-site.xml at org.apache.hadoop.mapred.InputFormatBase.listFiles(InputFormatBase.ja va:90) at org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.ja va:100) at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:1 30) at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java :204) at java.lang.Thread.run(Thread.java:595) 060325 002300 Server connection on port 50020 from 192.168.1.12: exiting > mapred input and output dirs must be absolute > --------------------------------------------- > > Key: HADOOP-52 > URL: http://issues.apache.org/jira/browse/HADOOP-52 > Project: Hadoop > Type: Bug > Components: mapred > Versions: 0.1 > Reporter: Doug Cutting > Assignee: Owen O'Malley > Fix For: 0.1 > Attachments: cwd.patch, cwd2.patch, cwd3.patch > > DFS converts relative pathnames to be under /user/$USER. But MapReduce jobs may be submitted by a different user than is running the jobtracker and tasktracker. Thus relative paths must be resolved before a job is submitted, so that only absolute paths are seen on the job tracker and tasktracker. I think the simplest way to fix this is to make JobConf.setInputDir(), setOutputDir(), etc. resolve relative pathnames. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira