falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pallavi Rao" <pallavi....@inmobi.com>
Subject Re: Review Request 33015: In a secure cluster, feed replication fails because of Authentication issues
Date Fri, 10 Apr 2015 04:19:28 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33015/#review79633
-----------------------------------------------------------



oozie/src/main/java/org/apache/falcon/oozie/feed/FSReplicationWorkflowBuilder.java
<https://reviews.apache.org/r/33015/#comment129132>

    Isn't it sufficient that only this action has the "mapreduce.job.hdfs-servers" property
set. Neither the pre-process nor the post-process access the source clusters, isn't it?
    
    Ideally, this should have been part of global config of the workflow. But, that is blocked
on OOZIE-2030.


- Pallavi Rao


On April 9, 2015, 2:21 p.m., Venkat Ranganathan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33015/
> -----------------------------------------------------------
> 
> (Updated April 9, 2015, 2:21 p.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-1149
>     https://issues.apache.org/jira/browse/FALCON-1149
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> In a Feed replication, the Oozie Java action executes on the target and accesses the
source cluster read only endpoint (which is a webhdfs endpoint). The Java action does not
have the delegation tokens for the source cluster populated in its UGI/conf and this results
in authentication failure
> 
> 
> Diffs
> -----
> 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FSReplicationWorkflowBuilder.java
6feb32e 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/FeedReplicationWorkflowBuilder.java
288e9de 
>   oozie/src/main/java/org/apache/falcon/oozie/feed/HCatReplicationWorkflowBuilder.java
30ca0a8 
> 
> Diff: https://reviews.apache.org/r/33015/diff/
> 
> 
> Testing
> -------
> 
> We did not catch this earlier as we were testing this on single node secure clusters
and there was a krb cache for the user (as part of the cluster setup) that allowed WebHDFSFS
to request delegation tokens as the target user.   Destroying the context allowed us to even
reproduce it in single node clusters
> 
> 
> Thanks,
> 
> Venkat Ranganathan
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message