beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Direct #442
Date Sun, 09 Sep 2018 00:04:21 GMT
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Direct/442/display/redirect>

------------------------------------------
[...truncated 226.81 KB...]
  0001  query:1; exportSummaryToBigQuery:true
  0002  query:2; exportSummaryToBigQuery:true
  0003  query:3; exportSummaryToBigQuery:true
  0004  query:4; exportSummaryToBigQuery:true; numEvents:10000
  0005  query:5; exportSummaryToBigQuery:true
  0006  query:6; exportSummaryToBigQuery:true; numEvents:10000
  0007  query:7; exportSummaryToBigQuery:true
  0008  query:8; exportSummaryToBigQuery:true
  0009  query:9; exportSummaryToBigQuery:true; numEvents:10000
  0010  query:10; exportSummaryToBigQuery:true
  0011  query:11; exportSummaryToBigQuery:true
  0012  query:12; exportSummaryToBigQuery:true

Performance:
  Conf  Runtime(sec)    (Baseline)  Events(/sec)    (Baseline)       Results    (Baseline)
  0000           6.7                     15033.1                      100000             

  0001           3.8                     26497.1                       92000             

  0002           1.5                     67430.9                         351             

  0003           2.6                     38865.1                         463             

  0004           1.6                      6215.0                          40             

  0005           4.4                     22722.1                          12             

  0006           1.1                      8952.6                         401             

  0007          14.1                      7087.2                           1             

  0008           1.7                     58788.9                        6000             

  0009           1.0                      9861.9                         298             

  0010          17.5                      5710.0                           1             

  0011           3.0                     33456.0                        1919             

  0012  *** not run ***
==========================================================================================

2018-09-09T00:04:14.202Z Generating 100000 events in batch mode
2018-09-09T00:04:16.345Z Waiting for main pipeline to 'finish'
2018-09-09T00:04:16.345Z DONE Query12

==========================================================================================
Run started 2018-09-09T00:03:09.889Z and ran for PT66.457S

Default configuration:
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}

Configurations:
  Conf  Description
  0000  query:0; exportSummaryToBigQuery:true
  0001  query:1; exportSummaryToBigQuery:true
  0002  query:2; exportSummaryToBigQuery:true
  0003  query:3; exportSummaryToBigQuery:true
  0004  query:4; exportSummaryToBigQuery:true; numEvents:10000
  0005  query:5; exportSummaryToBigQuery:true
  0006  query:6; exportSummaryToBigQuery:true; numEvents:10000
  0007  query:7; exportSummaryToBigQuery:true
  0008  query:8; exportSummaryToBigQuery:true
  0009  query:9; exportSummaryToBigQuery:true; numEvents:10000
  0010  query:10; exportSummaryToBigQuery:true
  0011  query:11; exportSummaryToBigQuery:true
  0012  query:12; exportSummaryToBigQuery:true

Performance:
  Conf  Runtime(sec)    (Baseline)  Events(/sec)    (Baseline)       Results    (Baseline)
  0000           6.7                     15033.1                      100000             

  0001           3.8                     26497.1                       92000             

  0002           1.5                     67430.9                         351             

  0003           2.6                     38865.1                         463             

  0004           1.6                      6215.0                          40             

  0005           4.4                     22722.1                          12             

  0006           1.1                      8952.6                         401             

  0007          14.1                      7087.2                           1             

  0008           1.7                     58788.9                        6000             

  0009           1.0                      9861.9                         298             

  0010          17.5                      5710.0                           1             

  0011           3.0                     33456.0                        1919             

  0012           2.0                     50658.6                        1919             

==========================================================================================

Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$4 getTempFilePrefix
INFO: Writing BigQuery temporary files to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/
before loading them.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/103065a1-a481-48bd-8b9a-7314d580966b.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/ff492329-9d15-4e4b-b11c-b446d06401bf.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/29e5cc5b-afcf-467c-b3dc-cf9226c3664f.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/0139e6cd-9f1e-45ce-bcec-24612bd11e2a.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/96323827-2392-443a-abcb-d5b7590e7833.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/00f54110-41aa-4e50-ab24-a32313c480ea.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/343e2b9d-5bfd-426e-8f19-5e7f8adaae98.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/a915f6f5-fe15-462a-b0a3-e5c1a0fafea2.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/8d0f96f8-d7b3-42b7-af37-b39ca795c59b.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/a23381df-3993-458c-bdc0-26e551ffb0ac.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/45b41786-f4cf-427a-bc96-b0a3d5bbc308.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/41fb43dd-403a-4a24-9a0c-c53bd9977e7b.
Sep 09, 2018 12:04:16 AM org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter <init>
INFO: Opening TableRowWriter to gs://temp-storage-for-perf-tests/nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/ef6b7e2d-109d-421c-a9b8-a98fbafe3f60.
Sep 09, 2018 12:04:18 AM org.apache.beam.sdk.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler
handleResponse
WARNING: Request failed with code 410, performed 0 retries due to IOExceptions, performed
1 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller
responsible for retrying): https://www.googleapis.com/upload/storage/v1/b/temp-storage-for-perf-tests/o?name=nexmark/BigQueryWriteTemp/beam_load_mainjenkins09090004167c407d41_4750caf0c4c8457e9cca5af42681e553/a23381df-3993-458c-bdc0-26e551ffb0ac&uploadType=resumable&upload_id=AEnB2UoPckuyDfQXeb0G9h0srGMFTqTGiLry5b9PP0DIYL87Lk0T-MsfNedQJaEoSyiToL042dO6KGRvE0oIsCaq7dPeCnvKZg
Sep 09, 2018 12:04:18 AM com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation
call
SEVERE: Exception not convertible into handled response
com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
  "code" : 503,
  "errors" : [ {
    "domain" : "global",
    "message" : "Backend Error",
    "reason" : "backendError"
  } ],
  "message" : "Backend Error"
}
	at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
	at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)


==========================================================================================
Run started 2018-09-09T00:03:09.889Z and ran for PT68.390S

Default configuration:
{"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1}
Exception in thread "main" 
Configurations:
  Conf  Description
  0000  query:0; exportSummaryToBigQuery:true
  0001  query:1; exportSummaryToBigQuery:true
  0002  query:2; exportSummaryToBigQuery:true
  0003  query:3; exportSummaryToBigQuery:true
  0004  query:4; exportSummaryToBigQuery:true; numEvents:10000
  0005  query:5; exportSummaryToBigQuery:true
  0006  query:6; exportSummaryToBigQuery:true; numEvents:10000
  0007  query:7; exportSummaryToBigQuery:true
  0008  query:8; exportSummaryToBigQuery:true
  0009  query:9; exportSummaryToBigQuery:true; numEvents:10000
  0010  query:10; exportSummaryToBigQuery:true
  0011  query:11; exportSummaryToBigQuery:true
  0012  query:12; exportSummaryToBigQuery:true

Performance:
  Conf  Runtime(sec)    (Baseline)  Events(/sec)    (Baseline)       Results    (Baseline)
  0000           6.7                     15033.1                      100000             

  0001           3.8                     26497.1                       92000             

  0002           1.5                     67430.9                         351             

  0003           2.6                     38865.1                         463             

  0004           1.6                      6215.0                          40             

org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.io.IOException: Failed to close
some writers
  0005           4.4                     22722.1                          12             

  0006           1.1                      8952.6                         401             

	at org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:332)
  0007          14.1                      7087.2                           1             

	at org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:302)
  0008           1.7                     58788.9                        6000             

	at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:197)
  0009           1.0                      9861.9                         298             

	at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:64)
  0010          17.5                      5710.0                           1             

	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
  0011           3.0                     33456.0                        1919             

	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
  0012           2.0                     50658.6                        1919             

	at org.apache.beam.sdk.nexmark.Main.savePerfsToBigQuery(Main.java:250)
==========================================================================================
	at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:162)

	at org.apache.beam.sdk.nexmark.Main.main(Main.java:477)
Caused by: java.io.IOException: Failed to close some writers
	at org.apache.beam.sdk.io.gcp.bigquery.WriteBundlesToFiles.finishBundle(WriteBundlesToFiles.java:248)
	Suppressed: java.io.IOException: com.google.api.client.googleapis.json.GoogleJsonResponseException:
410 Gone
{
  "code" : 503,
  "errors" : [ {
    "domain" : "global",
    "message" : "Backend Error",
    "reason" : "backendError"
  } ],
  "message" : "Backend Error"
}
		at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:432)
		at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:287)
		at org.apache.beam.sdk.io.gcp.bigquery.TableRowWriter.close(TableRowWriter.java:81)
		at org.apache.beam.sdk.io.gcp.bigquery.WriteBundlesToFiles.finishBundle(WriteBundlesToFiles.java:242)
		at org.apache.beam.sdk.io.gcp.bigquery.WriteBundlesToFiles$DoFnInvoker.invokeFinishBundle(Unknown
Source)
		at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.finishBundle(SimpleDoFnRunner.java:285)
		at org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimplePushbackSideInputDoFnRunner.finishBundle(SimplePushbackSideInputDoFnRunner.java:118)
		at org.apache.beam.runners.direct.ParDoEvaluator.finishBundle(ParDoEvaluator.java:225)
		at org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.finishBundle(DoFnLifecycleManagerRemovingTransformEvaluator.java:74)
		at org.apache.beam.runners.direct.DirectTransformExecutor.finishBundle(DirectTransformExecutor.java:188)
		at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:126)
		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
		at java.util.concurrent.FutureTask.run(FutureTask.java:266)
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
		at java.lang.Thread.run(Thread.java:748)
	Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
  "code" : 503,
  "errors" : [ {
    "domain" : "global",
    "message" : "Backend Error",
    "reason" : "backendError"
  } ],
  "message" : "Backend Error"
}
		at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
		at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
		at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
		at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
		at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
		at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
		at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
		... 4 more

> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 9,5,main]) completed. Took
1 mins 9.563 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' finished with non-zero
exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log
output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 4m 11s
62 actionable tasks: 58 executed, 4 from cache

Publishing build scan...
https://gradle.com/s/wofwandnakdyy

Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure

Mime
View raw message