gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (GEARPUMP-355) AppMasterResolver fails to run against a kerberized Hadoop cluster
Date Fri, 13 Oct 2017 02:24:01 GMT


ASF GitHub Bot commented on GEARPUMP-355:

Github user huafengw commented on the issue:
    Hi @titikakatoo, thanks for your response. I'd like to know how you set up your cluster.
I just got a single-node secured yarn cluster but found the command you posted goes well.

    So how many nodes are there in your cluster and where do you execute the yarnClient command,
just one of the node among them? And also, is there any special config I should notice? 

> AppMasterResolver fails to run against a kerberized Hadoop cluster
> ------------------------------------------------------------------
>                 Key: GEARPUMP-355
>                 URL:
>             Project: Apache Gearpump
>          Issue Type: Bug
>          Components: security, yarn
>    Affects Versions: 0.8.4
>            Reporter: Timea Magyar
>             Fix For: 0.8.4
> When trying to launch a Gearpump cluster in a kerberized Hadoop/Yarn environment, after
the Application Master address has been resolved as a prerequisite, the YarnAppMaster (responsible
for starting GearPump masters, workers, UI servers as Yarn containers) address (actor reference)
must be obtained via Kerberos/Spnego. (Kerberos over http)
> The current implementation for this resides in the AppMasterResolver class and is using
an apache http client (version 3.x) for establishing a connection to the Application Master
and obtain the above YarnAppMaster actor reference. Since the apache http client does not
support the negotiate authentication scheme in version 3.x (required for a connection over
kerberos/spnego) this step will always fail in a kerberized Yarn/Hadoop cluster set-up.
> I tested this in a secured/kerberized CDH 5.7.5 environment.  I would like to provide
a patch for this  by adapting the SPNEGO-enabled Hadoop web connection code from WebHDFS.

This message was sent by Atlassian JIRA

View raw message