beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dariusz Aniszewski (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-4041) Performance tests fail due to kubernetes load balancer problems
Date Fri, 18 May 2018 14:45:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480754#comment-16480754
] 

Dariusz Aniszewski commented on BEAM-4041:
------------------------------------------

Since those issues are still appearing, and [this PR to PerfKit|https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/1641] was
merged, I submitted [PR 5425|https://github.com/apache/beam/pull/5425] to increase retry
timeout from 3 to 6 minutes - please check it and merge it if it makes sense. If increasing
timeout is pointless please discard the PR.

> Performance tests fail due to kubernetes load balancer problems
> ---------------------------------------------------------------
>
>                 Key: BEAM-4041
>                 URL: https://issues.apache.org/jira/browse/BEAM-4041
>             Project: Beam
>          Issue Type: Bug
>          Components: testing
>            Reporter: Łukasz Gajowy
>            Assignee: Jason Kuster
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Recently, as we added more IOITs to be run on jenkins using kubernetes, some of them
started to fail randomly, because they couldn't retrieve LoadBalancer address. Normally obtaining
the address took about one minute. Perfkit waits for the address (actively checking for
it) for 3 minutes. This should be enough for getting the address, yet it recently started
to exceed the 3 minutes limit. I also noticed that this error didn't happen when there were
fewer tests.
> Example logs:
> https://builds.apache.org/view/A-D/view/Beam/job/beam_PerformanceTests_Compressed_TextIOIT_HDFS/31/console



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message