Return-Path: 
 <mapreduce-user-return-5377-apmail-hadoop-mapreduce-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id BF4E29AA8
	for <apmail-hadoop-mapreduce-user-archive@minotaur.apache.org>;
 Thu, 16 Aug 2012 00:48:17 +0000 (UTC)
Received: (qmail 83215 invoked by uid 500); 16 Aug 2012 00:48:15 -0000
Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org
Received: (qmail 83142 invoked by uid 500); 16 Aug 2012 00:48:15 -0000
Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:mapreduce-user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:mapreduce-user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:mapreduce-user@hadoop.apache.org>
List-Id: <mapreduce-user.hadoop.apache.org>
Reply-To: mapreduce-user@hadoop.apache.org
Delivered-To: mailing list mapreduce-user@hadoop.apache.org
Received: (qmail 83115 invoked by uid 99); 16 Aug 2012 00:48:15 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 16 Aug 2012 00:48:15 +0000
X-ASF-Spam-Status: No, hits=2.2 required=5.0
	tests=HTML_MESSAGE,SPF_HELO_PASS,SPF_PASS,WEIRD_PORT
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of jieli@cs.duke.edu designates
 152.3.140.1 as permitted sender)
Received: from [152.3.140.1] (HELO duke.cs.duke.edu) (152.3.140.1)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 16 Aug 2012 00:48:08 +0000
Received: from mail-ob0-f176.google.com (mail-ob0-f176.google.com
 [209.85.214.176])
	(authenticated bits=0)
	by duke.cs.duke.edu (8.14.5/8.14.5) with ESMTP id q7G0lkAu026292
	(version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=FAIL);
	Wed, 15 Aug 2012 20:47:46 -0400 (EDT)
X-DKIM: Sendmail DKIM Filter v2.8.3 duke.cs.duke.edu q7G0lkAu026292
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail;
	t=1345078067; bh=Nx4d2l8mXSVUfmvvA27o0DPs9yqoNQEhXtaTHIIfKd8=;
	h=MIME-Version:Date:Message-ID:Subject:From:To:Content-Type;
	b=pLcUWN1x259axRuDjuazPznwCr5m1/LWEvuK+ZmS7nv289P5c11IzL9+36HjV6E5I
	 06tgyuKrwShb2J/zaXN4f0+/YotTbPbVd5w+3vZ6zq/UqomlMDeDYQPsXX5A4JbB95
	 cVu46+h3fW+ByyvOcN7gZecC7vrDOtcp5J5532anN19m3owNmCHKZMq/QChRz62xXH
	 NJxQlijEA45rq6YItg8FGvUoAL9/nX/5AF1lwxYvqlHW+7MMZXAae5QcAWtsCznqUz
	 6KzJc7ap5+19cKKA1+XU8RFsYWg9t8INS3i2FyuASwZyOi0t+bsvJ9iYfjjE4Rk2LA
	 BiuPn/WUwW+5Q==
Received: by obbtb18 with SMTP id tb18so3681423obb.35
        for <multiple recipients>; Wed, 15 Aug 2012 17:47:46 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.50.184.138 with SMTP id eu10mr314431igc.68.1345078065938; Wed,
 15 Aug 2012 17:47:45 -0700 (PDT)
Received: by 10.231.63.212 with HTTP; Wed, 15 Aug 2012 17:47:45 -0700 (PDT)
Date: Wed, 15 Aug 2012 17:47:45 -0700
Message-ID: 
 <CALY2=u5VogZPStEdYZyrMu9kK4ySa=V5xg2M4ns1kMsKJT5mDQ@mail.gmail.com>
Subject: Hadoop profiling will fail when Kerberos is enabled
From: Jie Li <jieli@cs.duke.edu>
To: common-user@hadoop.apache.org, core-user@hadoop.apache.org,
        mapreduce-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=14dae9340f1b50c29a04c75764f5

--14dae9340f1b50c29a04c75764f5
Content-Type: text/plain; charset=ISO-8859-1

Hi all,

With Kerbero enabled, the job client is not able to download profiles from
the tasktracker. The following command will fail:

hadoop jar hadoop-examples-1.0.2.jar wordcount -Dmapred.task.profile=true
input output

The error message:

12/08/16 00:15:37 INFO mapred.JobClient: Running job: job_201208152133_0007
12/08/16 00:15:38 INFO mapred.JobClient:  map 0% reduce 0%
java.io.IOException: Server returned HTTP response code: 401 for URL:
http://datanode:50060/tasklog?plaintext=true&attemptid=attempt_201208152133_0007_m_000002_0&filter=profile<http://esv4-hcl266.grid.linkedin.com:50060/tasklog?plaintext=true&attemptid=attempt_201208152133_0007_m_000002_0&filter=profile>
    at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
    at
org.apache.hadoop.mapred.JobClient.downloadProfile(JobClient.java:1237)
    at
org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(JobClient.java:1315)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:533)
    at org.apache.hadoop.examples.WordCount.main(WordCount.java:67)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

Note if we copy the URL link above to the browser, we are able to see the
profile output. Any hint is appreciated!

Thanks,
Jie

--14dae9340f1b50c29a04c75764f5
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi all,<br><br>With Kerbero enabled, the job client is not able to download=
 profiles from the tasktracker. The following command will fail:<br>
<br>hadoop jar hadoop-examples-1.0.2.jar wordcount -Dmapred.task.profile=3D=
true input output<br><div id=3D":97"><br>The error message:<br><br>12/08/16=
 00:15:37 INFO mapred.JobClient: Running job: job_201208152133_0007<br>
12/08/16 00:15:38 INFO mapred.JobClient:=A0 map 0% reduce 0%<br>java.io.IOE=
xception: Server returned HTTP response code: 401 for URL: <a href=3D"http:=
//esv4-hcl266.grid.linkedin.com:50060/tasklog?plaintext=3Dtrue&amp;attempti=
d=3Dattempt_201208152133_0007_m_000002_0&amp;filter=3Dprofile" target=3D"_b=
lank">http://datanode:50060/tasklog?plaintext=3Dtrue&amp;attemptid=3Dattemp=
t_201208152133_0007_m_000002_0&amp;filter=3Dprofile</a><br>

=A0=A0=A0 at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Htt=
pURLConnection.java:1436)<br>=A0=A0=A0 at org.apache.hadoop.mapred.JobClien=
t.downloadProfile(JobClient.java:1237)<br>=A0=A0=A0 at org.apache.hadoop.ma=
pred.JobClient.monitorAndPrintJob(JobClient.java:1315)<br>

=A0=A0=A0 at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:533=
)<br>=A0=A0=A0 at org.apache.hadoop.examples.WordCount.main(WordCount.java:=
67)<br>=A0=A0=A0 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Met=
hod)<br>=A0=A0=A0 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMeth=
odAccessorImpl.java:39)<br>

=A0=A0=A0 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth=
odAccessorImpl.java:25)<br>=A0=A0=A0 at java.lang.reflect.Method.invoke(Met=
hod.java:597)<br>=A0=A0=A0 at org.apache.hadoop.util.ProgramDriver$ProgramD=
escription.invoke(ProgramDriver.java:68)<br>

=A0=A0=A0 at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java=
:139)<br>=A0=A0=A0 at org.apache.hadoop.examples.ExampleDriver.main(Example=
Driver.java:64)<br>=A0=A0=A0 at sun.reflect.NativeMethodAccessorImpl.invoke=
0(Native Method)<br>

=A0=A0=A0 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess=
orImpl.java:39)<br>=A0=A0=A0 at sun.reflect.DelegatingMethodAccessorImpl.in=
voke(DelegatingMethodAccessorImpl.java:25)<br>=A0=A0=A0 at java.lang.reflec=
t.Method.invoke(Method.java:597)<br>

=A0=A0=A0 at org.apache.hadoop.util.RunJar.main(RunJar.java:156)<br><br>Not=
e if we copy the URL link above to the browser, we are able to see the prof=
ile output. Any hint is appreciated!<br></div><br>Thanks,<br>Jie<br>

--14dae9340f1b50c29a04c75764f5--