Mailing-List: contact commits-help@beam.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@beam.apache.org
Date: Mon, 24 Apr 2017 17:46:04 +0000 (UTC)
From: "Davor Bonaci (JIRA)" <jira@apache.org>
To: commits@beam.apache.org
Message-ID: <JIRA.13066156.1492943888000.35000.1493055964114@Atlassian.JIRA>
In-Reply-To: <JIRA.13066156.1492943888000@Atlassian.JIRA>
References: <JIRA.13066156.1492943888000@Atlassian.JIRA> <JIRA.13066156.1492943888346@jira-lw-us.apache.org>
Subject: [jira] [Commented] (BEAM-2056) Add tests for exporting Beam Metrics
 to Flink Metrics
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 24 Apr 2017 17:46:08 -0000


    [ https://issues.apache.org/jira/browse/BEAM-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15981566#comment-15981566 ] 

Davor Bonaci commented on BEAM-2056:
------------------------------------

(fixing accidental change)

> Add tests for exporting Beam Metrics to Flink Metrics
> -----------------------------------------------------
>
>                 Key: BEAM-2056
>                 URL: https://issues.apache.org/jira/browse/BEAM-2056
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-flink
>            Reporter: Aljoscha Krettek
>            Assignee: Aljoscha Krettek
>             Fix For: First stable release
>
>
> There are currently no tests that verify that metrics that are reported using the Beam Metrics API are forwarded to Flink and a {{MetricReporter}}.
> A test for this would have to manually configure a Flink "Mini Cluster", as in
> {code}
> // start also a re-usable Flink mini cluster
> flink = new LocalFlinkMiniCluster(getFlinkConfiguration(), false);
> flink.start();
> flinkPort = flink.getLeaderRPCPort();
> {code}
> with {{getFlinkConfiguration()}}:
> {code}
> protected static Configuration getFlinkConfiguration() {
>     Configuration flinkConfig = new Configuration();
>     flinkConfig.setInteger(ConfigConstants.LOCAL_NUMBER_TASK_MANAGER, 1);
>     flinkConfig.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, 8);
>     flinkConfig.setInteger(ConfigConstants.TASK_MANAGER_MEMORY_SIZE_KEY, 16);
>     flinkConfig.setString(ConfigConstants.RESTART_STRATEGY_FIXED_DELAY_DELAY, "0 s");
>     flinkConfig.setString(ConfigConstants.METRICS_REPORTERS_LIST, "my_reporter");
>     flinkConfig.setString(ConfigConstants.METRICS_REPORTER_PREFIX + "my_reporter." + ConfigConstants.METRICS_REPORTER_CLASS_SUFFIX, MyTestReporter.class.getName());
>     return flinkConfig;
> }
> {code}
> where {{MyTestReporter}} is a {{MetricReporter}} that stores metrics being reported to it so we can verify that they are there after the job finishes.
> Running a Pipeline on the mini cluster should be possible by specifying "localhost" and the port we received as a cluster endpoint.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)