gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Timea Magyar (JIRA)" <>
Subject [jira] [Created] (GEARPUMP-355) AppMasterResolver fails to run against a kerberized Hadoop cluster
Date Wed, 11 Oct 2017 09:25:00 GMT
Timea Magyar created GEARPUMP-355:

             Summary: AppMasterResolver fails to run against a kerberized Hadoop cluster
                 Key: GEARPUMP-355
             Project: Apache Gearpump
          Issue Type: Bug
          Components: security, yarn
    Affects Versions: 0.8.4
            Reporter: Timea Magyar
             Fix For: 0.8.4

When trying to launch a Gearpump cluster in a kerberized Hadoop/Yarn environment, after the
Application Master address has been resolved as a prerequisite, the YarnAppMaster (responsible
for starting GearPump masters, workers, UI servers as Yarn containers) address (actor reference)
must be obtained via Kerberos/Spnego. (Kerberos over http)
The current implementation for this resides in the AppMasterResolver class and is using an
apache http client (version 3.x) for establishing a connection to the Application Master and
obtain the above YarnAppMaster actor reference. Since the apache http client does not support
the negotiate authentication scheme in version 3.x (required for a connection over kerberos/spnego)
this step will always fail in a kerberized Yarn/Hadoop cluster set-up.
I tested this in a secured/kerberized CDH 5.7.5 environment.  I would like to provide a patch
for this  by adapting the SPNEGO-enabled Hadoop web connection code from WebHDFS.

This message was sent by Atlassian JIRA

View raw message