gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (GEARPUMP-355) AppMasterResolver fails to run against a kerberized Hadoop cluster
Date Thu, 19 Oct 2017 05:25:00 GMT


ASF GitHub Bot commented on GEARPUMP-355:

Github user manuzhang commented on a diff in the pull request:
    --- Diff: experiments/yarn/src/main/scala/org/apache/gearpump/experiments/yarn/client/AppMasterResolver.scala
    @@ -75,3 +64,38 @@ class AppMasterResolver(yarnClient: YarnClient, system: ActorSystem)
    +object AppMasterResolver {
    +  val LOG = LogUtil.getLogger(getClass)
    +  def resolveAppMasterAddress(report: ApplicationReport, system: ActorSystem): ActorRef
= {
    +    val appMasterPath = s"${report.getOriginalTrackingUrl}" + "supervisor-actor-path"
    --- End diff --
    missing "/" here

> AppMasterResolver fails to run against a kerberized Hadoop cluster
> ------------------------------------------------------------------
>                 Key: GEARPUMP-355
>                 URL:
>             Project: Apache Gearpump
>          Issue Type: Bug
>          Components: security, yarn
>    Affects Versions: 0.8.4
>            Reporter: Timea Magyar
>             Fix For: 0.8.4
> When trying to launch a Gearpump cluster in a kerberized Hadoop/Yarn environment, after
the Application Master address has been resolved as a prerequisite, the YarnAppMaster (responsible
for starting GearPump masters, workers, UI servers as Yarn containers) address (actor reference)
must be obtained via Kerberos/Spnego. (Kerberos over http)
> The current implementation for this resides in the AppMasterResolver class and is using
an apache http client (version 3.x) for establishing a connection to the Application Master
and obtain the above YarnAppMaster actor reference. Since the apache http client does not
support the negotiate authentication scheme in version 3.x (required for a connection over
kerberos/spnego) this step will always fail in a kerberized Yarn/Hadoop cluster set-up.
> I tested this in a secured/kerberized CDH 5.7.5 environment.  I would like to provide
a patch for this  by adapting the SPNEGO-enabled Hadoop web connection code from WebHDFS.

This message was sent by Atlassian JIRA

View raw message