falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Venkat R <verama...@yahoo.com.INVALID>
Subject Re: Replication Job throws GSSException
Date Mon, 14 Jul 2014 18:42:38 GMT
Arpit,

Yes, I have copied the hadoop cluster config under oozie/conf/hadoop-conf-cluster-1 as explained
in the JIRA. However, I need to add the following line (correction to my previous email)
in the oozie/conf/hadoop-conf-cluster-1/mapred-site.xml:

 <property>
     <name>mapreduce.job.hdfs-servers</name>
     <value>webhdfs://gridAnn01.grid.linkedin.com,webhdfs://girdBnn01.grid.linkedin.com</value>
  </property>

this enabled gridB to do webhdfs calls to gridA and vice-versa. In the absence, i get authentication
errors. It seems Oozie needs to get tokens for both the clusters before it can kick off the
Falcon job that does the distcp.

Thanks
Venkat

On Monday, July 14, 2014 9:17 AM, Arpit Gupta <arpit@hortonworks.com> wrote:
 


hmm i dont recall having to set that. However take a look at the first
comment here

https://issues.apache.org/jira/browse/FALCON-389

I had to make sure NN and RM properties were set but that was when we had
to do hcat replication.



On Sun, Jul 13, 2014 at 10:28 PM, Venkat R <veramacha@yahoo.com.invalid>
wrote:

> After much of debugging, we had to add the following property to oozie's
> hadoop confs in order to obtain kerberos tokens between secure hadoop
> clusters. May be this can be added to the generated oozie job?
>
> <property>
 <name>oozie.launcher.mapreduce.job.hdfs-servers</name>
> <value>${nameNode1},${nameNode2}</value>
> </property>
>
> Venkat
>
>
> On Friday, July 11, 2014 11:34 AM, Venkat R <veramacha@yahoo.com.INVALID>
> wrote:
>
>
>
> Venkatesh,
>
> * Switched to using webhdfs
> * Using Hadoop-2/Yarn security enables
> * both source and target oozie-site.xml points to oozie/conf/hadoop-conf
> of all the clusters it talkes to.
>
> Venkat
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message