flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9235) Add Integration test for Flink-Yarn-Kerberos integration for flip-6
Date Mon, 07 May 2018 07:52:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465577#comment-16465577
] 

ASF GitHub Bot commented on FLINK-9235:
---------------------------------------

Github user zentol commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5953#discussion_r186350834
  
    --- Diff: flink-yarn-tests/src/test/java/org/apache/flink/yarn/YARNSessionFIFOITCase.java
---
    @@ -85,21 +91,56 @@ public void checkForProhibitedLogContents() {
     	public void testDetachedMode() throws InterruptedException, IOException {
     		LOG.info("Starting testDetachedMode()");
     		addTestAppender(FlinkYarnSessionCli.class, Level.INFO);
    -		Runner runner =
    -			startWithArgs(new String[]{"-j", flinkUberjar.getAbsolutePath(),
    -						"-t", flinkLibFolder.getAbsolutePath(),
    -						"-n", "1",
    -						"-jm", "768",
    -						"-tm", "1024",
    -						"--name", "MyCustomName", // test setting a custom name
    -						"--detached"},
    +
    +		File exampleJarLocation = getTestJarPath("StreamingWordCount.jar");
    +		// get temporary file for reading input data for wordcount example
    +		File tmpInFile = tmp.newFile();
    +		FileUtils.writeStringToFile(tmpInFile, WordCountData.TEXT);
    +
    +		ArrayList<String> args = new ArrayList<>();
    +		args.add("-j"); args.add(flinkUberjar.getAbsolutePath());
    +		args.add("-t"); args.add(flinkLibFolder.getAbsolutePath());
    +		args.add("-n"); args.add("1");
    +		args.add("-jm"); args.add("768");
    +		args.add("-tm"); args.add("1024");
    +		if (SecureTestEnvironment.getTestKeytab() != null) {
    +			args.add("-D" + SecurityOptions.KERBEROS_LOGIN_KEYTAB.key() + "=" + SecureTestEnvironment.getTestKeytab());
    +		}
    +		if (SecureTestEnvironment.getHadoopServicePrincipal() != null) {
    +			args.add("-D" + SecurityOptions.KERBEROS_LOGIN_PRINCIPAL.key() + "=" + SecureTestEnvironment.getHadoopServicePrincipal());
    +		}
    +		args.add("--name"); args.add("MyCustomName");
    +		args.add("--detached");
    +		Runner clusterRunner =
    +			startWithArgs(
    +				args.toArray(new String[args.size()]),
     				"Flink JobManager is now running on", RunTypes.YARN_SESSION);
     
     		// before checking any strings outputted by the CLI, first give it time to return
    -		runner.join();
    -		checkForLogString("The Flink YARN client has been started in detached mode");
    +		clusterRunner.join();
     
     		if (!isNewMode) {
    +			checkForLogString("The Flink YARN client has been started in detached mode");
    +
    +			// in legacy mode we have to wait until the TMs are up until we can submit the job
    +			LOG.info("Waiting until two containers are running");
    +			// wait until two containers are running
    +			while (getRunningContainers() < 2) {
    --- End diff --
    
    there a comment at L 151: `// additional sleep for the JM/TM to start and establish connection`,
the job execution should probably happen after that...


> Add Integration test for Flink-Yarn-Kerberos integration for flip-6
> -------------------------------------------------------------------
>
>                 Key: FLINK-9235
>                 URL: https://issues.apache.org/jira/browse/FLINK-9235
>             Project: Flink
>          Issue Type: Test
>    Affects Versions: 1.5.0
>            Reporter: Shuyi Chen
>            Assignee: Shuyi Chen
>            Priority: Major
>
> We need to provide an integration test for flip-6 similar to YARNSessionFIFOSecuredITCase
for the legacy deployment mode.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message