beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1033) WindowedWordCountIT is flaky
Date Thu, 01 Dec 2016 07:11:58 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15711129#comment-15711129
] 

ASF GitHub Bot commented on BEAM-1033:
--------------------------------------

GitHub user markflyhigh opened a pull request:

    https://github.com/apache/incubator-beam/pull/1479

    [BEAM-1033] Bigquery Verifier Retry When Query Failed

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
     - [x] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [x] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [ ] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
    
    ---
    
     - Add retry when query job failed.
     - check QueryResponse.jobComplete instead of rows. reference from https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/query#response

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/markflyhigh/incubator-beam Add-retry-when-bq-response-is-failed

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/1479.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1479
    
----
commit 6c5a07f1949146d3fc4aed7fa1cfc55863fcdfb2
Author: Mark Liu <markliu@markliu-macbookpro.roam.corp.google.com>
Date:   2016-12-01T06:20:12Z

    [BEAM-1033] Bigquery Verifier Retry When Query Failed

----


> WindowedWordCountIT is flaky
> ----------------------------
>
>                 Key: BEAM-1033
>                 URL: https://issues.apache.org/jira/browse/BEAM-1033
>             Project: Beam
>          Issue Type: Bug
>          Components: testing
>            Reporter: Pei He
>            Assignee: Mark Liu
>
> Jenkins link:
> https://builds.apache.org/job/beam_PreCommit_MavenVerify/5145/console
> Running org.apache.beam.examples.WindowedWordCountIT
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 304.282 sec <<<
FAILURE! - in org.apache.beam.examples.WindowedWordCountIT
> testWindowedWordCountInBatch(org.apache.beam.examples.WindowedWordCountIT)  Time elapsed:
304.282 sec  <<< FAILURE!
> java.lang.AssertionError: 
> Expected: Expected checksum is (cd5b52939257e12428a9fa085c32a84dd209b180)
>      but: Invalid BigQuery response: {"jobComplete":false,"jobReference":{"jobId":"job_0STNX_OD83tQOzo6MvmqXCrk61U","projectId":"apache-beam-testing"},"kind":"bigquery#queryResponse"}
> 	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
> 	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
> 	at org.apache.beam.runners.dataflow.testing.TestDataflowRunner.run(TestDataflowRunner.java:164)
> 	at org.apache.beam.runners.dataflow.testing.TestDataflowRunner.run(TestDataflowRunner.java:93)
> 	at org.apache.beam.runners.dataflow.testing.TestDataflowRunner.run(TestDataflowRunner.java:61)
> 	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:179)
> 	at org.apache.beam.examples.WindowedWordCount.main(WindowedWordCount.java:224)
> 	at org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountPipeline(WindowedWordCountIT.java:88)
> 	at org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatch(WindowedWordCountIT.java:59)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> 	at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:393)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Results :
> Failed tests: 
>   WindowedWordCountIT.testWindowedWordCountInBatch:59->testWindowedWordCountPipeline:88

> Expected: Expected checksum is (cd5b52939257e12428a9fa085c32a84dd209b180)
>      but: Invalid BigQuery response: {"jobComplete":false,"jobReference":{"jobId":"job_0STNX_OD83tQOzo6MvmqXCrk61U","projectId":"apache-beam-testing"},"kind":"bigquery#queryResponse"}
> Tests run: 3, Failures: 1, Errors: 0, Skipped: 0



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message