Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Date: Thu, 21 Dec 2017 11:54:00 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@flink.apache.org
Message-ID: <JIRA.13111730.1508857679000.510864.1513857240058@Atlassian.JIRA>
In-Reply-To: <JIRA.13111730.1508857679000@Atlassian.JIRA>
References: <JIRA.13111730.1508857679000@Atlassian.JIRA> <JIRA.13111730.1508857679292@jira-lw-us.apache.org>
Subject: [jira] [Commented] (FLINK-7909) Unify cluster creation for test
 bases
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 21 Dec 2017 11:54:04 -0000


    [ https://issues.apache.org/jira/browse/FLINK-7909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299909#comment-16299909 ] 

ASF GitHub Bot commented on FLINK-7909:
---------------------------------------

Github user GJL commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4896#discussion_r158259149
  
    --- Diff: flink-connectors/flink-connector-filesystem/src/test/java/org/apache/flink/streaming/connectors/fs/RollingSinkSecuredITCase.java ---
    @@ -215,23 +211,15 @@ private static void startSecureFlinkClusterWithRecoveryModeEnabled() {
     			dfs.mkdirs(new Path("/flink/checkpoints"));
     			dfs.mkdirs(new Path("/flink/recovery"));
     
    -			org.apache.flink.configuration.Configuration config = new org.apache.flink.configuration.Configuration();
    -
    -			config.setInteger(ConfigConstants.LOCAL_NUMBER_TASK_MANAGER, 1);
    -			config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, DEFAULT_PARALLELISM);
    -			config.setBoolean(ConfigConstants.LOCAL_START_WEBSERVER, false);
    -			config.setInteger(ConfigConstants.LOCAL_NUMBER_JOB_MANAGER, 3);
    -			config.setString(HighAvailabilityOptions.HA_MODE, "zookeeper");
    -			config.setString(CoreOptions.STATE_BACKEND, "filesystem");
    -			config.setString(HighAvailabilityOptions.HA_ZOOKEEPER_CHECKPOINTS_PATH, hdfsURI + "/flink/checkpoints");
    -			config.setString(HighAvailabilityOptions.HA_STORAGE_PATH, hdfsURI + "/flink/recovery");
    -			config.setString("state.backend.fs.checkpointdir", hdfsURI + "/flink/checkpoints");
    -
    -			SecureTestEnvironment.populateFlinkSecureConfigurations(config);
    -
    -			cluster = TestBaseUtils.startCluster(config, false);
    -			TestStreamEnvironment.setAsContext(cluster, DEFAULT_PARALLELISM);
    +			MINICLUSTER_CONFIGURATION.setBoolean(ConfigConstants.LOCAL_START_WEBSERVER, false);
    --- End diff --
    
    Because of this:
    ```
    private static void skipIfHadoopVersionIsNotAppropriate() {
    		// Skips all tests if the Hadoop version doesn't match
    		String hadoopVersionString = VersionInfo.getVersion();
    		String[] split = hadoopVersionString.split("\\.");
    		if (split.length != 3) {
    			throw new IllegalStateException("Hadoop version was not of format 'X.X.X': " + hadoopVersionString);
    		}
    		Assume.assumeTrue(
    			// check whether we're running Hadoop version >= 3.x.x
    			Integer.parseInt(split[0]) >= 3
    		);
    	}
    ```
    I assume that the test will never run.
    
    I wonder if the test has ever worked correctly.


> Unify cluster creation for test bases
> -------------------------------------
>
>                 Key: FLINK-7909
>                 URL: https://issues.apache.org/jira/browse/FLINK-7909
>             Project: Flink
>          Issue Type: Improvement
>          Components: Tests
>    Affects Versions: 1.4.0
>            Reporter: Till Rohrmann
>            Assignee: Till Rohrmann
>
> Flink contains different test bases, e.g. {{AbstractTestBase}}, {{MultipleProgramsTestBase}}, {{JavaProgramTestBase}} and the {{StreamingMultipleProgramsTestBase}}. Some of them start resources ({{FlinkMiniCluster}}) automatically, while others expose a start method. Some test bases set the {{ExecutionEnvironment}} while others use the {{FlinkMiniCluster}} directly. In order to make things more maintainable I propose to unify these test bases a bit and introduce an {{ExternalResource}} which encapsulates the resource start up and shut down.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)