hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jie Li <ji...@cs.duke.edu>
Subject Hadoop profiling will fail when Kerberos is enabled
Date Thu, 16 Aug 2012 00:47:45 GMT
Hi all,

With Kerbero enabled, the job client is not able to download profiles from
the tasktracker. The following command will fail:

hadoop jar hadoop-examples-1.0.2.jar wordcount -Dmapred.task.profile=true
input output

The error message:

12/08/16 00:15:37 INFO mapred.JobClient: Running job: job_201208152133_0007
12/08/16 00:15:38 INFO mapred.JobClient:  map 0% reduce 0%
java.io.IOException: Server returned HTTP response code: 401 for URL:
http://datanode:50060/tasklog?plaintext=true&attemptid=attempt_201208152133_0007_m_000002_0&filter=profile<http://esv4-hcl266.grid.linkedin.com:50060/tasklog?plaintext=true&attemptid=attempt_201208152133_0007_m_000002_0&filter=profile>
    at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
    at
org.apache.hadoop.mapred.JobClient.downloadProfile(JobClient.java:1237)
    at
org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(JobClient.java:1315)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:533)
    at org.apache.hadoop.examples.WordCount.main(WordCount.java:67)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

Note if we copy the URL link above to the browser, we are able to see the
profile output. Any hint is appreciated!

Thanks,
Jie

Mime
View raw message