Mailing-List: contact dev-help@gearpump.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@gearpump.incubator.apache.org
Date: Wed, 11 Oct 2017 10:11:00 +0000 (UTC)
From: "Timea Magyar (JIRA)" <jira@apache.org>
To: dev@gearpump.incubator.apache.org
Message-ID: <JIRA.13108550.1507716645000.26696.1507716660833@Atlassian.JIRA>
In-Reply-To: <JIRA.13108550.1507716645000@Atlassian.JIRA>
References: <JIRA.13108550.1507716645000@Atlassian.JIRA> <JIRA.13108550.1507716645947@jira-lw-us.apache.org>
Subject: [jira] [Created] (GEARPUMP-356) Launch of Gearpump on Hadoop/Yarn
 not possible of cluster protected by firewall
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 11 Oct 2017 10:11:04 -0000

Timea Magyar created GEARPUMP-356:
-------------------------------------

             Summary: Launch of Gearpump on Hadoop/Yarn not possible of cluster protected by firewall
                 Key: GEARPUMP-356
                 URL: https://issues.apache.org/jira/browse/GEARPUMP-356
             Project: Apache Gearpump
          Issue Type: Bug
          Components: hadoop, security, yarn
    Affects Versions: 0.8.4
            Reporter: Timea Magyar
             Fix For: 0.8.4


Launching Gearpump on a Hadoop/Yarn cluster protected by a firewall from an edge node outside of the cluster communication will fail.
When launching Gearpump / submitting a Gearpump application the client side residing on the above mentioned edge node has to establish communication with 4 components:

1. the YarnAppMaster in order to be able to retrieve/resolve an active configuration file (via tcp)
2. the Gearpump master(s) (via tcp)
3. the JarStoreServer (via http) to transmit the application jar file to be submitted
4. the Gearpump AppMaster of the respective gearpump application to be submitted to send in tasks/requests

In all four cases the ports used by the above mentioned components have to be open, otherwise communication will fail. 
Currently the config parameter "akka.remote.netty.tcp.port" is set to 0, meaning that all ports assigned for an actorsystem defined in Gearpump are random and can be anything between  1024-65535
There is no configuration based way to set the a fixed port in the above four cases.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)