hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lucian Iordache <lucian.george.iorda...@gmail.com>
Subject Re: Job works only when TaskTracker & JobTracker on the same machine
Date Wed, 18 May 2011 09:00:23 GMT
Hi Todd,

I think the cluster is well configured, the HDFS and HBase work fine in
distributed mode (several datanodes and regionservers are started and work
correctly).
- all the slaves are present in the ../conf/slaves file
- the jobtracker host and port are well set in mapred-site on the
TaskTracker machine

I know that it must be some configuration problem, but I don't manage to
find out which is it.
By the way, the problem appeared after the upgrade of the cluster from
CDH3B4 to CDH3U0

Thanks,
Lucian

On Tue, May 17, 2011 at 8:42 PM, Todd Lipcon <todd@cloudera.com> wrote:

> Hi Lucian,
>
> Sounds like you didn't configure MapReduce to run on top of HDFS? If you
> want to run on a distributed cluster, you need a distributed file system set
> up as well.
>
> -Todd
>
>
> On Mon, May 16, 2011 at 8:31 AM, Lucian Iordache <
> lucian.george.iordache@gmail.com> wrote:
>
>> Hello,
>>
>> I have a Hadoop/Hbase cluster, cloudera cdh3u0 version, with several
>> machines. I've created a job that does some work using the information from
>> a HBase table.
>>
>> The next example *works* fine:
>> - start JobTracker
>> - start TaskTracker *on the same machine* as the JobTracker
>> - run the job
>> - the job completes successfully
>>
>> The next example *does not work*:
>> - start JobTracker
>> - start TaskTracker *on another machine* (not the same as the JobTracker)
>> - run the job
>> - I get the next error in the console:
>> 11/04/21 12:21:01 INFO mapred.JobClient: Running job:
>> job_201104201820_0007
>> 11/04/21 12:21:02 INFO mapred.JobClient: map 0% reduce 0%
>> 11/04/21 12:21:02 INFO mapred.JobClient: Task Id :
>> attempt_201104201820_0007_m_000002_0, Status : FAILED
>> Error initializing attempt_201104201820_0007_m_000002_0:
>> java.io.FileNotFoundException: File
>> file:/var/ui/hadoop/mapred/system/job_201104201820_0007/jobToken does not
>> exist.
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383)
>> at
>> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>> at
>> org.apache.hadoop.mapred.TaskTracker.localizeJobTokenFile(TaskTracker.java:4033)
>> at
>> org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1148)
>> at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1089)
>> at
>> org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2257)
>> at
>> org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2221)
>> 11/04/21 12:21:02 WARN mapred.JobClient: Error reading task outputhttp://
>> cene.ro.schlund.net:50060/tasklog?plaintext=true&attemptid=attempt_201104201820_0007_m_000002_0&filter=stdout
>> 11/04/21 12:21:02 WARN mapred.JobClient: Error reading task outputhttp://
>> cene.ro.schlund.net:50060/tasklog?plaintext=true&attemptid=attempt_201104201820_0007_m_000002_0&filter=stderr
>> 11/04/21 12:21:02 INFO mapred.JobClient: Task Id :
>> attempt_201104201820_0007_r_000002_0, Status : FAILED
>>
>> Can you help me with this problem?
>>
>> Best Regards,
>> Lucian
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Mime
View raw message