hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Calling the JobTracker from Reducer throws InvalidCredentials GSSException
Date Sun, 29 Sep 2013 10:51:30 GMT
Hm, I think I forgot the bit where Oozie also adds a DT for itself to
use: https://github.com/apache/oozie/blob/release-4.0.0/core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java#L374.
Doing that same additional thing to the job's driver, works just fine
for me.

On Sun, Sep 29, 2013 at 10:51 AM, Manish Verma
<manish.lifepart2@gmail.com> wrote:
> Hi Harsh,
>
> Thanks for the pointer but it didn't fix my problem. Basically I am trying
> to read the counters set by the mapper in my reducer. And the reducer has
> trouble reading the mapper's counter using the JobClient. I tested this
> logic in a single node cloudera VM which did not have kerberos installed.
>
> Thanks
> Manish
>
>
> On Sat, Sep 28, 2013 at 11:06 AM, Harsh J <harsh@cloudera.com> wrote:
>>
>> You'll need to reuse the security tokens of the current job to
>> communicate without requiring a re-auth from the task. See the
>> specific code snippet (of what Oozie does for similar goals) at
>>
>> https://github.com/apache/oozie/blob/release-4.0.0/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/MapReduceMain.java#L90
>> and also see MAPREDUCE-4324.
>>
>> On Sat, Sep 28, 2013 at 2:43 AM, Manish Verma
>> <manish.lifepart2@gmail.com> wrote:
>> > I am trying to get the job tracker counters in my reducer. It works on
>> > single node demo hadoop but fails on a real cluster where kerberos is
>> > used
>> > for authentication.
>> >
>> > ********************************************
>> > RunningJob parentJob =
>> >                     client.getJob(JobID.forName(
>> > context.getConfiguration().get("mapred.job.id") ));
>> >
>> > Counters counters = parentJob.getCounters();
>> > ********************************************
>> >
>> > The call to getCounter() API throws GSSException (No valid credentials
>> > provided - Failed to find any kerberos tgt).
>> >
>> > I launched this job using hadoop jar command.
>> >
>> > Any help would be much appreciated.
>> >
>> > Thanks
>> > Manish
>> >
>>
>>
>>
>> --
>> Harsh J
>
>



-- 
Harsh J

Mime
View raw message