From commits-return-55627-archive-asf-public=cust-asf.ponee.io@beam.apache.org Thu Jan 11 19:45:55 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 957CA180656 for ; Thu, 11 Jan 2018 19:45:55 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 852C5160C23; Thu, 11 Jan 2018 18:45:55 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 09177160C13 for ; Thu, 11 Jan 2018 19:45:52 +0100 (CET) Received: (qmail 8245 invoked by uid 500); 11 Jan 2018 18:45:52 -0000 Mailing-List: contact commits-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.apache.org Delivered-To: mailing list commits@beam.apache.org Received: (qmail 8236 invoked by uid 99); 11 Jan 2018 18:45:52 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Jan 2018 18:45:52 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 8E6251A087C for ; Thu, 11 Jan 2018 18:45:51 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.59 X-Spam-Level: * X-Spam-Status: No, score=1.59 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, KAM_NUMSUBJECT=0.5, RCVD_IN_DNSWL_LOW=-0.7, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id bz6XI3dClydB for ; Thu, 11 Jan 2018 18:45:36 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 2623B5F250 for ; Thu, 11 Jan 2018 18:45:35 +0000 (UTC) Received: from jenkins-master.apache.org (unknown [62.210.60.235]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id D4DD2E0383 for ; Thu, 11 Jan 2018 18:45:33 +0000 (UTC) Received: from jenkins-master.apache.org (localhost [127.0.0.1]) by jenkins-master.apache.org (ASF Mail Server at jenkins-master.apache.org) with ESMTP id 771B6A001E for ; Thu, 11 Jan 2018 18:45:31 +0000 (UTC) Date: Thu, 11 Jan 2018 18:44:44 +0000 (UTC) From: Apache Jenkins Server To: commits@beam.apache.org Message-ID: <409896624.1857.1515696331483.JavaMail.jenkins@jenkins-master.apache.org> In-Reply-To: <936423122.1817.1515692615584.JavaMail.jenkins@jenkins-master.apache.org> References: <936423122.1817.1515692615584.JavaMail.jenkins@jenkins-master.apache.org> Subject: Build failed in Jenkins: beam_PerformanceTests_JDBC #218 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkqVKZPv7YyHBB3FvWfV7XQehwe/Ga3aadzSNknt8g382X3uN8A3SOQ+Ixq9HxS+ZlN6XR4TECySmSRy2JN5Rx8svxAD0TjtSF9LuU98dD+LniNDP7Lq6gvRFuJhbMHoS0nuTizDZLsK4X8TW5MyV9w+jFbdoZfRE5O/Mse0fkOeL5uoIS/3Vvu/W+x9QSjDkB7CaU56bPFlQjqqJBl3Cn9r34CkXQZYnLb/NjW4vcpw0+TgMUAPTIVEr5BTPZRshz19g7huwg3zANT5HBIZnzV4hsVY9w4JHkceFdKi/ibNnjPjsFs9pm0HSGJ/RDxjIvSTYT02eH4+m1RAYaj2E9QIDAQAB X-Jenkins-Job: beam_PerformanceTests_JDBC X-Jenkins-Result: FAILURE See Changes: [robertwb] Fix typo in variable name: window_fn --> windowfn [iemejia] [BEAM-2764] Change document size range to fix flakiness on SolrIO tests [lcwik] Add fn API progress reporting protos [iemejia] [BEAM-2787] Fix MongoDbIO exception when trying to write empty bundle [lcwik] Upgrade snappy-java to version 1.1.4 [lcwik] Upgrade slf4j to version 1.7.25 [chamikara] Updates bigtable.version to 1.0.0-pre3. [lcwik] Add ThrowingBiConsumer to the set of functional interfaces [altay] Added snippet tags for documentation [altay] Added concrete example for CoGroupByKey snippet [jbonofre] Add license-maven-plugin and some default license merges [relax] BEAM-934 Fixed build by fixing firebug error. [relax] BEAM-934 Enabled firebug after fixing the bug. [relax] BEAM-934 Fixed code after review. [jbonofre] [BEAM-2328] Add TikaIO [mingmxu] Add DSLs module [mingmxu] [BEAM-301] Initial skeleton for Beam SQL [mingmxu] checkstyle and rename package [mingmxu] redesign BeamSqlExpression to execute Calcite SQL expression. [mingmxu] Fix inconsistent mapping for SQL FLOAT [mingmxu] [BEAM-2158] Implement the arithmetic operators [mingmxu] [BEAM-2161] Add support for String operators [mingmxu] [BEAM-2079] Support TextIO as SQL source/sink [mingmxu] [BEAM-2006] window support Add support for aggregation: global, HOP, [mingmxu] Support common-used aggregation functions in SQL, including: [mingmxu] [BEAM-2195] Implement conditional operator (CASE) [mingmxu] [BEAM-2234] Change return type of buildBeamPipeline to [mingmxu] [BEAM-2149] Improved kafka table implemention. [mingmxu] support UDF [mingmxu] update JavaDoc. [mingmxu] [BEAM-2255] Implement ORDER BY [mingmxu] [BEAM-2288] Refine DSL interface as design doc of BEAM-2010: 1. rename [mingmxu] [BEAM-2292] Add BeamPCollectionTable to create table from [mingmxu] fix NoSuchFieldException [mingmxu] [BEAM-2309] Implement VALUES and add support for data type CHAR (to be [mingmxu] [BEAM-2310] Support encoding/decoding of TIME and new DECIMAL data type [mingmxu] DSL interface for Beam SQL [mingmxu] [BEAM-2329] Add ABS and SQRT math functions [mingmxu] upgrade to version 2.1.0-SNAPSHOT [mingmxu] rename SQL to Sql in class name [mingmxu] [BEAM-2247] Implement date functions in SQL DSL [mingmxu] [BEAM-2325] Support Set operator: intersect & except [mingmxu] Add ROUND function on DSL_SQL branch. [mingmxu] register table for both BeamSql.simpleQuery and BeamSql.query [mingmxu] Add NOT operator on DSL_SQL branch (plus some refactoring) [mingmxu] [BEAM-2444] BeamSql: use java standard exception [mingmxu] [BEAM-2442] BeamSql surface api test. [mingmxu] [BEAM-2440] BeamSql: reduce visibility [mingmxu] Remove unused BeamPipelineCreator class [mingmxu] [BEAM-2443] apply AutoValue to BeamSqlRecordType [mingmxu] Update filter/project/aggregation tests to use BeamSql [mingmxu] Remove UnsupportedOperationVisitor, which is currently just a no-op [mingmxu] restrict the scope of BeamSqlEnv [mingmxu] Add ACOS, ASIN, ATAN, COS, COT, DEGREES, RADIANS, SIN, TAN, SIGN, LN, [mingmxu] [BEAM-2477] BeamAggregationRel should use Combine.perKey instead of [mingmxu] use static table name PCOLLECTION in BeamSql.simpleQuery. [mingmxu] Small fixes to make the example run in a runner agnostic way: - Add [mingmxu] [BEAM-2193] Implement FULL, INNER, and OUTER JOIN: - FULL and INNER [mingmxu] UDAF support: - Adds an abstract class BeamSqlUdaf for defining Calcite [mingmxu] BeamSql: refactor the MockedBeamSqlTable and related tests [mingmxu] MockedBeamSqlTable -> MockedBoundedTable [mingmxu] Test unsupported/invalid cases in DSL tests. [mingmxu] [BEAM-2550] add UnitTest for JOIN in DSL [mingmxu] support TUMBLE/HOP/SESSION _START function [mingmxu] Test queries on unbounded PCollections with BeamSql DSL API. Also add [mingmxu] [BEAM-2564] add integration test for string functions [mingmxu] CAST operator supporting numeric, date and timestamp types [mingmxu] POWER function [mingmxu] support UDF/UDAF in BeamSql [mingmxu] upgrade pom to 2.2.0-SNAPSHOT [mingmxu] [BEAM-2560] Add integration test for arithmetic operators. [mingmxu] cleanup BeamSqlRow [mingmxu] proposal for new UDF [mingmxu] [BEAM-2562] Add integration test for logical operators [mingmxu] [BEAM-2384] CEIL, FLOOR, TRUNCATE, PI, ATAN2 math functions [mingmxu] [BEAM-2561] add integration test for date functions [mingmxu] refactor the datetime test to use ExpressionChecker and fix [mingmxu] [BEAM-2565] add integration test for conditional functions [mingmxu] rebased, add RAND/RAND_INTEGER [mingmxu] [BEAM-2621] BeamSqlRecordType -> BeamSqlRowType [mingmxu] [BEAM-2563] Add integration test for math operators Misc: 1. no SQRT in [mingmxu] [BEAM-2613] add integration test for comparison operators [mingmxu] remove README.md and update usages in BeamSqlExample [mingmxu] update pom.xml to fix conflict [mingmxu] remove maven-jar-plugin [mingmxu] move dsls/sql to sdks/java/extensions/sql [mingmxu] rename package org.apache.beam.dsls.sql to [mingmxu] move all implementation classes/packages into impl package [mingmxu] move BeamRecord to sdk/core [mingmxu] use BitSet for nullFields [mingmxu] refactor BeamRecord, BeamRecordType, BeamSqlRecordType, BeamRecordCoder [mingmxu] Remove redundant windowing information from the BeamRecord itself [mingmxu] [rebased] remove nullFields in BeamRecord [mingmxu] [BEAM-2733] update javadoc for BeamSql [mingmxu] [BEAM-2742] change Field type from primitives to boxed types [mingmxu] [BEAM-2730] make BeamRecord an immutable type [mingmxu] [BEAM-2745] add BeamRecordSqlType.getFieldTypeByIndex() [mingmxu] Update BeamSqlExample: - Fix mvn example - Add aggregation/group by [mingmxu] take SerializableFunction as UDF. [mingmxu] [BEAM-2744] rename BeamRecordType#size() [mingmxu] update JavaDoc for BeamRecord, BeamRecordType. Also only create new UDF [mingmxu] take CombineFn as UDAF. [mingmxu] [BEAM-2740] Hide BeamSqlEnv. [mingmxu] Add @Experimental and ImmutableList.copyOf [mingmxu] Switch to unmodifiableList [mingmxu] rename simpleQuery to query and query to queryMulti [mingmxu] update JavaDoc of BeamSortRel [mingmxu] var_pop and var_samp [robertwb] Fix (and test) FnApiRunner in grpc mode. [lcwik] [BEAM-2862] Add PTransform overrides specific to execution of [klk] Key StateTable off id, not full StateTag [ekirpichov] Cleans up PTransform validation across Beam [relax] Add ability to set job labels to BEAM Java SDK [chamikara] [BEAM-2861] Fix tests which fail with google credential [chamikara] Add first few BigtableWriteException to suppressed list when rethrowing [mingmxu] [BEAM-2804] support TIMESTAMP in sort [chamikara] Fix GqlQueryTranslateFn to pass localhost option to DatastoreFactory [klk] Improve error message for bad DoFn URN in ParDoTranslation [klk] Key DoFnInstanceManager cache on AppliedPTransform [klk] Add fast path to ParDoTranslation for known ParDo class [millsd] Mark AuctionOrBidWindowCoder as consistentWithEquals [owenzhang1990] [BEAM-2632] Use Junit Paramaterized test suits in TextIOReadTest [staslevin] [BEAM-2859] Fixed processing timers not being properly fired when [millsd] Enforce correctness of consistentWithEquals and fix broken hashCode() [millsd] Make AuctionOrBidWindowCoder use structuralValue instead of [pei] flink-runner: constructs exception string only when neccessary, it [klk] Key FlinkRunner streaming translation off URN [kirpichov] [BEAM-1637] Create Elasticsearch IO compatible with ES 5.x [kirpichov] Piggyback: deflakes MongoDBGridFSIOTest which failed during merge [aljoscha.krettek] [BEAM-2807] Fix NPE error on CoderTypeSerializerConfigSnapshot [lcwik] [BEAM-2790] Use byte[] instead of ByteBuffer to read from [lcwik] Fall back to inputStream.read(byteBuffer) in case of off-heap byteBuffer [chamikara] Improve performance of bundle retry [lcwik] [BEAM-2956] Attempt to correctly report the Dataflow distribution in GCP [lcwik] [BEAM-2961] Add missing OnTimeBehavior proto translation to [lcwik] [BEAM-2808] Improving error message in [echauchot] [BEAM-2957] Fix flaky ElasticsearchIOTest.testSplit in [kirpichov] Speeds up CompressedSourceTest [klk] Move DoFnInfo to SDK util [klk] Update Dataflow container version to 20170918 [aljoscha.krettek] [BEAM-2948] Register default FileSystems in StreamOperator.setup() [robertwb] [BEAM-2964] Exclude incompatible six release. [kirpichov] [BEAM-407] Fixes findbugs warnings in OffsetRangeTracker [robertwb] Refactor fn api runner into universal local runner. [robertwb] Implement ULR subprocess mode. [robertwb] Allow worker to be started as a subprocess. [robertwb] Streaming Job API. [robertwb] Lint and documentation. [robertwb] Allow subclasses of tuple, list, and dict as pvaluish inputs/outputs. [robertwb] Support multiple materializations of the same pvalue. [kirpichov] Updates grpc-google-pubsub-v1 to grpc-google-cloud-pubsub-v1 [kirpichov] Marks TikaIO as experimental to allow backward-incompatible changes [aljoscha.krettek] [BEAM-2377] Allow cross compilation (2.10,2.11) for flink runner [tgroh] Add PipelineOptionsTranslation [tgroh] Add Nullable getters to MetricsContainerImpl [robertwb] Exclude incompatible six release, part 2 [robertwb] Enable progress request handling in python SDK harness [robertwb] Minor cleanup. [robertwb] Revert "[BEAM-2377] Allow cross compilation (2.10,2.11) for flink [robertwb] Execute windowing in Fn API runner. [klk] Send portable ParDo protos to Dataflow instead of just DoFnInfo [klk] Update Dataflow worker to 20170921 [tgroh] Clarify semantics of objects returned by state access [klk] DataflowRunner depends on, and shades, protobuf [klk] Shade generated Runner API classes in Dataflow runner [tgroh] Initial set of pipeline jobs. [relax] Make sure that we default to alwaysRetry instead of passing in a null [relax] Add unit-test coverage. [relax] Strip table decorators before creating tables. [relax] Move stripping code into BigQueryHelpers and add better unit-test [klk] Revert "This closes #3859: [BEAM-2884] Send portable protos for ParDo in [relax] Fix type parameter in AvroIO.Write [chamikara] Revert "Initial set of pipeline jobs." [klk] Update Dataflow worker to 20170922-01 [tgroh] Artifact API Cleanup [tgroh] Add a Local FS implementation of the Artifact Staging API [chamikara] Included immediate results after CoGroupByKey for better readability in [tgroh] [BEAM-2876] Add preliminary provision API [chamikara] Updates WriteToBigQuery PTransform to get project id from [tgroh] Use the SubscriptionProvider in PubsubUnboundedSource [kirpichov] [BEAM-2467] Kinesis source watermark based on [tgroh] Update the Dataflow v1b3 API version [tgroh] Change checksum back to md5 [relax] Move file deletion into subsequent ParDo. [tgroh] Move Runners Core Fn Api classes into a package [chamikara] Add test to fix partial writouts after a bundle retry [github] Catch subprocess error in proto generation. [robertwb] Add some timing information to testing scripts. [robertwb] Use de(stributed)tox rather than plain old tox for Python testing. [ekirpichov] Sets a TTL on BigQueryIO.read().fromQuery() temp dataset [lcwik] [BEAM-2959] Fix proto enums to use "YYY_UNSPECIFIED" as the first [kirpichov] Removes codepaths for reading unsplit BigQuery sources [kirpichov] PAssert improvements. [kirpichov] Introduces Reshuffle.viaRandomKey() [kirpichov] Adds ReadableFile.toString() for debugging [lcwik] [BEAM-2606] make WindowFnTestUtils use the value in addition to the [lcwik] [BEAM-2997] Encapsulate enums within a message so that C++/Python have [robertwb] Avoid using beta grpc implementation. [chamikara] Option to disable validation of BigtableIO.write target table [robertwb] Updated Pipeline job for Jenkins PreCommit [jasonkuster] Allow users to skip Java or Python branch via a comment. Also use [jasonkuster] Modify trigger phrase to require specifying which language. [ekirpichov] Support for using raw avro records from BigQuery [jasonkuster] Introduce a more sustainable way to add new SDKs. [jasonkuster] Update variable references. [aljoscha.krettek] Adapt Flink StateInternals to new state semantics [ekirpichov] Fix "Writing data to multiple destinations" part of AvroIO javadoc [kirpichov] [BEAM-2455] Backlog size retrieval for Kinesis source [klk] Increase precommit timeout to 150 minutes [thw] BEAM-1037 Support for new State API in ApexRunner [chamikara] Updating BEAM_CONTAINER_VERSION for new worker [iemejia] Fix code style issues for HBaseIO [relax] update shade settings to handle Calcite dependencies [relax] add relocate in shading; add NOTICE and include Janino license; [bchambers] Change the state sampler to use structured names [lcwik] [BEAM-2989] Fixed error when using Void type in WithKeys. [lcwik] [BEAM-2958] Adding user agent string to PipelineOptions. [lcwik] [BEAM-3007] Add test which covers PCollection consumed by Flatten and [tgroh] Disable testReadPdfFile [robertwb] Use imports from grpc. [klk] Split Java and Python precommit jobs [github] Enabling phrase triggering of Python PreCommit [altay] [BEAM-2999] Split ValidatesRunner test from Python Postcommit [datcm] [BEAM-3018] Remove duplicated methods in StructuredCoder [robertwb] Enable discovery of log_handler tests. [tgroh] Add a Bare-bones ReferenceRunner Job Service [tgroh] Remove any_param field from FunctionSpec [tgroh] Add an Endpoints Proto file [tgroh] Fix ReferenceRunnerJobServer checkstyle [tgroh] Add an ArtifactServiceStager [chamikara] Adds API for defining Splittable DoFns. [klk] Make PackageUtil a proper class encapsulating its ExecutorService [klk] Use AutoValue for Dataflow PackageAttributes [klk] Refactor PackageUtil for more and simpler asynchrony [tgroh] [BEAM-2877][BEAM-2881] Add Java SDK harness container image and support [chamikara] OrderedCode library provides encoding a sequence of typed enities into a [lcwik] [BEAM-2701] ensure objectinputstream uses the right classloader for [klk] Limit Java postcommit to Java SDK and related [herohde] [BEAM-2877] Add presubmit for Go code [herohde] [BEAM-2877] Minor Go fixes for typos, proto changes and beamctl [herohde] Fix typo in python script [herohde] [BEAM-2877] Add netty-transport-native-epoll to Java harness uberjar [altay] whitelist time command in tox to prevent warning [herohde] [BEAM-3031] Use blocking grpc dial in Go unit tests [aljoscha.krettek] [BEAM-3027] Correctly set output type on SourceId-stripper [klk] Allow precommits to coexist with phrase triggering [klk] Restore Python precommit [klk] Enable phrase triggering of Java PreCommit [klk] Enable phrase triggering of Go PreCommit [klk] Add runners/direct-java to Java precommit [klk] Extend nightly build timeout to 4 hours [klk] Fix typo in nightly release script [kirpichov] Fixes TextIO and AvroIO tests of watchForNewFiles [kirpichov] Add @FunctionalInterface annotation to several interfaces [klk] Ensure metric names are not null or empty [klk] Relocate sdks.common.v1 and portability.v1 in Java direct runner [chamikara] Fixes a bug in query splitting. [relax] [maven-release-plugin] prepare branch release-2.2.0 [relax] [maven-release-plugin] prepare for next development iteration [relax] Bump Python version for next release. [klk] Only fail nightly release at end, for more detailed signal [klk] Only fail Java postcommit at end [valentyn] Pin runner harness container image for Dataflow distributions of BEAM [altay] [BEAM-2600] Add minimal python SDK harness container [altay] Revert "Use de(stributed)tox rather than plain old tox for Python [altay] Make linter happy. [tgroh] Move the Job Server into a Submodule [tgroh] Add Artifact Staging Endpoint to PrepareJobResponse [klk] Add model/ and submodules to Maven structure and gen_protos.py [klk] Move sdk/common/runner-api protos into model/pipeline and [klk] Move sdks/common/fn-api protos to model/fn-execution [klk] Remove sdks/common [klk] Re-namespace model/pipeline to model.pipeline [klk] Re-namespace model/job-management to model.job_management [klk] Re-namespace model/fn-execution to model.fn_execution [klk] Improve error message when standard_coders.yaml not found [klk] Dataflow runner: shade protobuf and beam model [herohde] [BEAM-2878] Use bintray as default container images [herohde] Add license to new file [lcwik] PipelineOptionsFactory should prevent non PipelineOptions interfaces [lcwik] [BEAM-3016] Fix blocking issue within run() when channel terminates [herohde] [BEAM-2576] Reflect portability proto changes in Go [herohde] Add licenses to generated code [altay] Improve the documentation for CombineFn [robertwb] Futurize stage 1 run (AUTOMATED futurize) [robertwb] Add futurize to pylint script [robertwb] re-order imports (AUTOMATED isort) [robertwb] Cleanup whitespace (AUTOMATED autopep8) [robertwb] Change the ptransform_test to use assertEqual rather than assertTrue for [robertwb] Add special work to handle indexable return types. Introduce a base [robertwb] Fix remaining style issues from auto conversion, switch ref in pylint to [robertwb] Restore license position that got screwed up [robertwb] Add future to deps and explicitly set isort req and update pylint in tox [robertwb] Change the xxx_todo_changeme[x]s to reasonable values [robertwb] Parallelize pylint [robertwb] Clarify docstring and update run_pylint to be more clear about what [robertwb] Re-order comment (reviewer feedback) [robertwb] Break out the exclusion list for futurize [robertwb] Change the examples with long-lambdas which were pulled out into defs to [robertwb] pr/cl-feedback from c-y-koo [robertwb] Fix snippets [tgroh] Increment dataflow client version [lcwik] [BEAM-3048] Remove RAND_RANGE in WindowedWordCount [altay] Add an option for dataflow job labels. [robertwb] Align names with those produced by the dataflow runner harness. [robertwb] Fix from any -> bytes transition. [robertwb] Add an element batching transform. [ekirpichov] Introduces Contextful [ekirpichov] Supports side inputs in MapElements and FlatMapElements [robertwb] Add progress metrics to Python SDK. [chamikara] Sets user agent in BigTableIO.Read.getBigTableService(). [kenn] Return null when timer not found instead of crashing [kirpichov] Add custom tempLocation support to BigQueryIO. [kenn] Allow checking timers set in TriggerStateMachineTester [kenn] Set end-of-window timer in AfterWatermarkStateMachine [kenn] Do not set EOW timer in TriggerStateMachineTester [lcwik] [BEAM-1487, BEAM-3016] Address termination correctness issues in [lcwik] [BEAM-3063] Improve VoidCoder structural value to use a single shared [tgroh] [BEAM-409] Avoiding integer division in ceil [kenn] Relocate everything shaded by Java SDK harness [tgroh] Remove "beta" from the Cancel Command Prefix [jbonofre] [BEAM-1017] Add RedisIO [rangadi] Kafka exactly-once sink. Tested manually with direct runner and on [rangadi] Disable other sink tests. Need to fix how tests use global [rangadi] 1) Add a note about metadata expiration. 2) Fetch number of partitions [rangadi] Add basic metrics. [rangadi] Input PCollection has the coder for elements. No need to ask for them [rangadi] Close producers after one minute of inactivity. This closes producers [rangadi] assign a random shard id once in setup(), rather than each time in [rangadi] Improve how we use MockProducer in tests. Remove global instance and use [rangadi] Support version 0.11 through ProducerSpEL (reflection). Tests work with [rangadi] Add runner check. Add withReadCommit() to reader. [rangadi] update JavaDoc for withEOS(). [rangadi] update JavaDoc for withEOS(). [rangadi] Add missing license header in ProducerSpEL.java. [kenn] Add RawPTransform.migrate(SdkComponents) for re-serialization [kenn] Add TransformPayloadTranslator.rehydrate to optionally specialize [kenn] Add custom rehydration for TestStream [kenn] Add custom rehydration for Combine [kenn] Add custom rehydration for ParDo [kenn] Add custom rehydration for WriteFiles [kenn] Better error message for RehydratedComponents.getCoder [kenn] Reinstate proto round trip in Java DirectRunner [kenn] Fix typo in UnboundedSource deserialization error message [kenn] Support side inputs in CombineTranslation [kenn] Add NotSerializable.forUrn to key by URN for non-serializable overrides [kenn] Clearer getOrDefault style in RehydratedComponents [altay] Wordcount on fnapi pipeline and IT test. [kenn] DirectRunner: Replace use of RawPTransform with NotSerializable.forUrn [tgroh] Update PipelineTest.testReplacedNames [rangadi] [BEAM-2720] Update kafka client version to 0.11.0.1 [kenn] Improve GcsFileSystem errors messages slightly [kenn] Add ability to stage explicit file list [kenn] Stage the portable pipeline in Dataflow [kenn] Stage the pipeline without using a temp file [kenn] Add assertion that valid jobs must have staged pipeline [kenn] Remove duplicate mocking in DataflowRunnerTest [valentyn] Pin runner harness also for official BEAM releases. [kenn] Add standalone version of seed job [altay] Created Java snippets file [kenn] Unit test to repro NPE in PTransformTranslation [kenn] Do not crash when RawPTransform has null spec [chamikara] Avoids generating proto files for Windows if grpcio-tools is not [chamikara] Adding lull tracking for python sampler [klk] Clone source to a distinguished subdirectory of Jenkins workspace [ekirpichov] [BEAM-2682] Deletes AvroIOTransformTest [kenn] Add dep on Apache-licensed findbugs-annotations implementation [kenn] Make Java core SDK root dir NonNull by default [kenn] NonNull by default in sdk/coders [kenn] NonNull by default in sdk/annotations [kenn] NonNull by default in sdk/runners [kenn] NonNull by default in sdk/state [kenn] NonNull by default for sdk/testing [kenn] Ignore findbugs in AutoValue generated classes [kenn] NonNull by default in metrics [altay] Unit test for label pipeline option [robertwb] Implement FnApi side inputs in Python. [github] Increase job_beam_PreCommit_Java_MavenInstall timeout from 2.5 to 4 [kenn] Make the main seed job standalone [kenn] Increase seed job(s) timeout to 30 minutes [kenn] Increase seed job(s) timeout to 60 minutes [kenn] Fix typo in seed jobs [kenn] Rename seed job so it is first in glob used by prior seed job [kenn] Fix typo in seed job [kenn] Rearrange .gitignore slightly [kenn] Fix RAT exclusions [kenn] Manage RAT plugin more centrally; only toggle skipping [kenn] Increase Java postcommit timeout to 240 [kenn] Fix remaining nullability errors [altay] Add zip to the list of accepted extra package file types. [klk] Stage the pipeline in Python DataflowRunner [herohde] [BEAM-3005] Add resource limits to provision proto [herohde] [BEAM-3005] Set JVM max heap size in java container [herohde] Declare .go and Dockerfile as text in gitattributes [herohde] Update rat exclusion for python and Go protos [herohde] CR: [BEAM-3005] Set JVM max heap size in java container [altay] Add python_requires to setup.py [lcwik] [BEAM-2566] Decouple SDK harness from Dataflow runner by elevating [lcwik] Add missing @RunWith to test. [kirpichov] Many improvements to TikaIO [kirpichov] Adds ParseResult.failure() [kenn] Remove coveralls invocations from all jobs [klk] Remove extraneous type arguments in PAssert [klk] Remove extraneous type arguments in Latest.java [klk] Suppress AutoValue warnings in TextIO [ekirpichov] Reading spanner schema transform [klk] NonNull by default in sdk/transforms [klk] NonNull by default in sdk/transforms/windowing [klk] NonNull by default in sdk/transforms/reflect [klk] NonNull by default in sdk/transforms/display [klk] NonNull by default in sdk/transforms/join [klk] NonNull by default in sdk/transforms/splittabledofn [klk] Temporarily disable Dataflow pipeline_url metadata [iemejia] [BEAM-3111] Upgrade Elasticsearch to 5.6.3 and clean pom [iemejia] [BEAM-3112] Improve error messages in ElasticsearchIO test utils [chamikara] [BEAM-2979] Fix a race condition in getWatermark() in KafkaIO. [kirpichov] [BEAM-1542] SpannerIO: mutation encoding and size estimation [chamikara] Added VoidSerializer for KafkaIO. Modified KafkaIO.Write.values() to [kirpichov] [BEAM-3054] Uses locale-insensitive number formatting in ESIO and tests [kirpichov] [BEAM-2468] Reading Kinesis records in the background [chamikara] Allows to set a Cloud Spanner host. [lcwik] Changed the mutation detector to be based on structural value only [lcwik] [BEAM-2482] - CodedValueMutationDetector should use the coders [tgroh] Update Website Precommits to new RVM instructions [lcwik] Remove obsolete extra parameter [chamikara] Updates Python datastore wordcount example to take a dataset parameter. [tgroh] Do not relocate generated Model Classes [kenn] Fix working dir in website precommits [mingmxu] [BEAM-2203] Implement TIMESTAMPADD [robertwb] Ensure producers are always set on unpickling from runner api protos. [tgroh] Add License Header to SqlTypeUtils [lcwik] Getting AutoValue 1.5.1 working in Beam. [chamikara] Updates BigQueryTableSource to consider data in streaming buffer when [altay] [BEAM-3135] Adding futures dependency to python SDK [tgroh] Add sdks/java/fn-execution [tgroh] Add a runners/java-fn-execution module [ekirpichov] [BEAM-2728] Extension for sketch-based statistics : HyperLogLog [tgroh] Add all portability protos to Go [tgroh] Fix Go package comment for syscallx [tgroh] [BEAM-3113] Disable stack trace optimization in java container [tgroh] [BEAM-3114] Generate text proto config properly in container boot code [ajamato] update dataflow.version [tgroh] Migrate shared Fn Execution code to Java7 [tgroh] Fix Repackaging Configuration in the the DirectRunner [chamikara] Added a preprocessing step to the Cloud Spanner sink. [kenn] Check that bigtableWriter is non-null before calling close(). [kenn] Stage the portable pipeline; put URL in pipeline options [xumingmingv] [BEAM-2203] Implement TIMESTAMPDIFF() [tgroh] Ban any Java SDK dependency in Fn Execution Libs [kenn] Fix syntax error in ReadSpannerSchema [kenn] Remove extraneous profiles from Python build [kenn] Remove extraneous runner profiles from non-IT steps of pipeline build [niemo] Fix BigTableIO to use non-deprecated Futures.addCallback function [xumingmingv] [BEAM-2203] Implement 'timestamp - interval' [echauchot] [BEAM-2853] remove Nexmark README.md now that the doc is in the website [chamikara] Added vcf file io source and modified _TextSource to optionally handle [chamikara] Ensure Kafka sink serializers are set. [mingmxu] [BEAM-2528] create table [robertwb] [BEAM-3040] Disable flaky subprocess variant of ULR test. [tgroh] Fork Control Clients into java-fn-execution [tgroh] Fork Data Service Libraries to java-fn-execution [ekirpichov] Adds logging at INFO for all creation, deletion and copying of files in [xumingmingv] change `withAllowedTimestampSkew` [kirpichov] [BEAM-3118] Fix thread leaks in ElasticsearchIOTest.testWriteWith* when [github] Set OnTimeBehavior in windowing strategy to FIRE_ALWAYS. [lcwik] Fix typo in GLOBAL_WINDOW_CODER urn constant. [kenn] Stage the portable pipeline; put URL in pipeline options [lcwik] Set save_main_session to True for csv module [lcwik] Remove redundant public modifier on inner interfaces [kenn] NonNull by default in sdk/io [kenn] NonNull by default in sdk/values [kenn] NonNull by default in sdk/io/fs [kenn] NonNull by default in sdk/io/range [kenn] NonNull by default in runners/core/construction [kenn] NonNull by default in runners/core/construction/metrics [kenn] NonNull by default in runners/core/triggers [kenn] NonNull by default in runners/core/fn [kenn] NonNull by default in runners/core/metrics [kenn] NonNull by default in runners/core [bchambers] Dataflow: Add option to upload heap dumps to GCS [lcwik] Add A FnService, FnServer to runner Fn Execution [chamikara] Properly handle side input exception when all reader threads complete [wcn] Introduce a property to influence output location. [chamikara] [BEAM-3060] Adds TextIOIT for DirectRunner and local filesystem [xumingmingv] support topicPartition in BeamKafkaTable [chamikara] Integrate bundle retry code for the DirectRunner [ccy] Fix flakiness in sideinputs_test [kenn] Fix expectations in CombineTest [kenn] Tidy a troublesome TestStreamTest [kenn] Truncate the very last fixed window if it goes beyond representable time [kenn] ReduceFnRunner: Do not manage EOW hold or timer, set GC hold and timer [holden] Flush stderr on failures [robertwb] [BEAM-3150] Fix threading for progress reporting in SDKHarness [robertwb] Update minimum cython version to 0.26.1 [kirpichov] WriteFiles improvements: [chamikara] Updated jdbc's pom.xml [lcwik] [BEAM-2926] Migrate to using a trivial multimap materialization within [lcwik] Rebase and update code to honor findbugs @Nullable/@Nonnull conversion [lcwik] Update Dataflow worker image to be compatible with side input changes. [lcwik] Replace the View.As transforms for Dataflow batch because the entire [lcwik] Fix BatchViewOverrides ViewAsSingleton to apply the combine fn that was [kedin] [Nexmark] Extract GeneratorCheckpoint into a separate class. Move [kedin] [Nexmark] Extract PersonGenerator, StringsGenerator, LongGenerator from [kedin] [Nexmark] Extract AuctionGenerator, PriceGenerator from Generator [kedin] [Nexmark] Extract BidGenerator from Generator [altay] Fixed adding timestamp and id attributes to pubsub messages [kirpichov] Specifies numShards on windowed writes examples, as it is now required [valentyn] Fix user_score example. [lcwik] [BEAM-3202] Ensure that PipelineOptions.getOptionsId is always [kirpichov] [BEAM-3035] Introduces Reify transform [iemejia] [BEAM-3206] Shut down executor when spark runner finishes [lcwik] Move AVRO files to avro source location [lcwik] Initial gradle build definitions for all java based modules which is [lcwik] Add license headers to all build.gradle files. [lcwik] Add provided scope plugin and migrate compileOnly/testCompileOnly to [lcwik] Add the ability to list dependencies recursively. Also ban the [lcwik] Update comments [lcwik] Configure the checkstyle plugin [lcwik] Add support to run the Apache Rat plugin Add gradle generated files to [lcwik] Adding Python gradle changes [lcwik] Fix python tasks to not re-use the default gradle lifecycle names. [lcwik] Add test task dependencies for testCython and testGcp and also add lint [lcwik] Create build_rules file moving out specific plugins to them. [lcwik] Add missing build_rules file. [lcwik] Ignore test categories NeedsRunner, ValidatesRunner, and all *IT tests [lcwik] Add code style validation and auto-formatting to Gradle build. [lcwik] Add beam-sdks-java-core dependency to beam-local-artifact-service [lcwik] Add gradle build for sdks/go [lcwik] Get evaluation dependency order fixed because of dependency on [lcwik] Update comments on root project [lcwik] Set beamUseDummyRunner = true [lcwik] Move spotless plugin to delegate config [lcwik] Fix spotless config [lcwik] Change hamcrest and junit to testCompile in [lcwik] Add findbugs to the build [lcwik] Add test to check phase [lcwik] Filter resources to set dataflow environrment versions in [lcwik] Force dependency resolution of common Java dependencies [lcwik] Filter sdk.properties, set pom.version and build timestamp [lcwik] Use dummy runner in tests in google-cloud-dataflow-java [lcwik] Add test dependecy on beam-sdks-java-core to [lcwik] Migrate findbugs to be configurable from the JavaNatureConfiguration [lcwik] Setup several maven-surefire-plugin configurations [lcwik] Remove added sdk-core dependencies because of [lcwik] Fix provided scope in sdks/java/extensions/sorter [lcwik] Ban hamcrest-all/mockito-all and migrate to hamcrest-core/mockito-core [lcwik] Add gradle rule for python container [lcwik] Fix python container gradle build [lcwik] Fix hadoop-file-system test dependencies Explain why hcatalog is failing [lcwik] Add license header to sdks/python/build.gradle [lcwik] Get spark runner unit tests to pass. [lcwik] Define the correct versions of apex-core Add comments to hcatalog/hbase [lcwik] Simplify ApexYarnLauncher matching Attempt to create dependency tree [lcwik] Enable configuraton on demand by default Get ApexYarnLauncher test to [lcwik] Fix sdks/java/core testCompile dependency to bring in [lcwik] Re-enable Python SDK [lcwik] Disable findbugs within sdks/java/io/kafka to match findbugs [lcwik] Add a task dependency visualization plugin. Resolve a spurious error [lcwik] Fix classpath bundling issue caused by Hive and its dependencies [lcwik] Define Tika dependencies version as variable [lcwik] Rebase on apache/master and add new subprojects, update versions of [lcwik] Skip test failures in hadoop related modules due to dependency [lcwik] Add Jacoco plugin [lcwik] Add Gradle builds for gcemd and gcsproxy [lcwik] Add Java SDK Container Gradle build file [lcwik] Create uber jar of the Java SDK harness and copy it into the location [lcwik] Make docker tasks correctly depend on required build tasks [lcwik] Fix runners/gcp/* go package paths [lcwik] Add shadow configuration for sdks/java/core updating all other projects [lcwik] Configure default shading profile on all java sub-projects to shade away [lcwik] Shade direct-java and sql projects. Update repackaged locations to [lcwik] Fix gcemd and gcsproxy builds. Fix whitespace formatting. [lcwik] Enforce that gogradle dependency resolution/installation is done [lcwik] Also ensure that resolving test dependencies are done sequentially with [lcwik] Use valid repackage path on shadow relocations Fix Spark runners tests [lcwik] Move user.avsc to directory which corresponds with namespace under [lcwik] Add comments to various parts of build_rules.gradle [lcwik] Fix excludes to correctly match the .idea/ directory. [lcwik] Update build.gradle files to correspond with pom.xml changes within [lcwik] Enable FFMP -> JavaCC path to generate Beam SQL parser for [tgroh] Add a Local Java Core Module [tgroh] Move StructuralKey to the local execution module [tgroh] Add A TransformExecutorFactory [chamikara] Clarify documentation on support for reading from/writing to time [lcwik] Enable Java Gradle Precommit in Jenkins [lcwik] Simplify maintainability by generating tasks and command line from list [lcwik] Limit the commit status length to 255 characters to honor Github limit [lcwik] Revert PR/3976, the Jenkins Gradle plugin expects the Gradle Wrapper to [lcwik] Sync with master and add new subproject for beam-runners-local-java-core [kirpichov] Improved validation of DynamicDestinations results in BigQueryIO.write [chamikara] Dataflow and PerfKit profiles; hash for 100.000.000 lines [lcwik] Re-apply PR/3976 and disable the Jenkins Gradle build to fix the other [lcwik] Update the location of the root build directory to coincide with the [robertwb] Add option to run PTransform steps from PipelineRunner [robertwb] Minor cleanup. [robertwb] Add logging and test aborting for timeouts. [lcwik] Add precommit integration tests to examples/java [lcwik] Update Gradle PreCommit to publish test results. [robertwb] Cleanup temp file handling in tests. [robertwb] Remove obsolete dependence of FnApiRunner on MapTaskExecutorRunner. [tgroh] Remove dupe dep (com.google.code.findbugs:jsr305) [altay] Bump Python SDK harness container version to use with Beam HEAD [lcwik] Update sdks/java/io/hadoop/jdk1.8-tests to run and pass for Gradle [altay] Update julia set example comments [lcwik] Move fn-execution org.apache.beam.harness.channel [lcwik] Move Common Fn Execution Concepts to fn-execution [lcwik] Fix test runtime class conflicts and get hbase tests passing. [jbonofre] Update scm to use gitbox.apache.org [dimon.ivan] Adds withInstanceId and withProjectId to the BigtableIO Read and Write [dimon.ivan] Removes call to .withBitableOptions() form JavaDoc. [luke_zhu] Allowed for declared state to be accessed as a supertype [luke_zhu] Edited an incorrect comment in the test case. [github] Removed trailing whitespace. [tgroh] Add a Logging Service implementation to FnExecution [tgroh] Implement an Slf4j Log Writer [tgroh] Update UNSPECIFIED level documentation [tgroh] Add an interface to execute Bundles [tgroh] Move outstanding work maintenance out of evaluateBundle [tgroh] Add an ExecutionDriver to local java [tgroh] Migrate MonitorRunnable into QuiescenceDriver [tgroh] Rename BundleExecutor to BundleProcessor [chamikara] [BEAM-3060] Add Compressed TextIOIT [valentyn] Fix repository path for Python Jenkins builds. Source directory was [github] Fix typo [github] Fix typo in proto files. [tgroh] Fork FnDataService from runners-core [tgroh] Remove core-java/fn package [kirpichov] [BEAM-3130] View.asMap() causes a ClassCastException in Apex runner [kenn] Add license header to SideInputTranslationTest [kenn] DataflowRunner: delegate @Setup and @Teardown in stateful ParDo [iemejia] Make ParDoLifecycleTest exception tests part of the ValidatesRunner set [iemejia] Disable ParDoLifecycleTest from the ValidatesRunner set on failing [tgroh] Add logging when Staging takes a long time [szewinho] Added support for multiple filesystems in TextIO [tgroh] Maintain In-flight preparation requests [robertwb] Also backoff on IO Exceptions. [szewinho] Typo in name [kirpichov] [BEAM-3247] fix Sample.any performance [bjchambers] Updating dataflow API version to newer release. [kirpichov] Fixes flaky error in JmsIOTest [xumingmingv] [BEAM-3238][SQL] Move Coders to map in BeamRecordSqlType [xumingmingv] [BEAM-3238][SQL] Add BeamRecordSqlTypeBuilder [altay] Use pip from currently executing Python. [iemejia] Fix missing type in View transform javadoc [iemejia] Make timeout longer for Spark ValidatesRunner tests [altay] preinstall various packages for better startup performance and [mingmxu] convert JOIN as LOOKUP [kirpichov] [BEAM-2870] Strips partition decorators when creating/patching tables in [msaul] Added loose failure mode to allow individual VCF record reads to fail [aaltay] Spell out isort arguments [github] Various worker cleanups. (#4195) [iemejia] [BEAM-3244] Ensure execution of teardown method on Flink's DoFnOperator [iemejia] [BEAM-3244] Enable PardoLifecycleTest for the Flink runner [mingmxu] add BeamSqlSeekableTable to SurfaceTest [altay] Add support for --save_main_session flag [dimon.ivan] - Adds ability to provide configuration function for BigtableOptions. - [szewinho] Temporary fix for failing tests on dataflow runner. Bug is described in [tgroh] Verify that the exposed endpoint stages artifacts [dimon.ivan] Fixes JavaDoc problem with @Override [wcn] Revert "Introduce a property to influence output location." [wcn] Introduce a property to influence output location. [kirpichov] Fixes a bug in Sample.Any [kirpichov] enforce fixed sharding [kirpichov] Merges Writer.openWindowed/Unwindowed and removes result of close() [kirpichov] non-null window/pane in FileResult [kirpichov] remove ShardAssignment [kirpichov] consolidates windowed/unwindowed finalize fns somewhat [kirpichov] Unifies windowed and unwindowed finalize. [kirpichov] Refactors WriteFiles into sub-transforms [kirpichov] Converts WriteFiles to AutoValue [kirpichov] Makes checkstyle and findbugs happy [kirpichov] Renames spilled back to unwritten [kirpichov] Fixes tests [kirpichov] Reintroduces dynamic sharding with windowed writes for bounded [kirpichov] Adds a deduplication key to Watch, and uses it to handle growing files [dimon.ivan] - Removes local instance related JavaDoc - Adds iinfo to toString and [dimon.ivan] Simplifies if condition around deciding apply or not optiosn [tgroh] Extract LogicalEndpoint to java-fn-execution [arostami] Remove a test case from VcfIO. [Pablo] Updating Dataflow Api protos [tgroh] Move StreamObserverFactory to java fn-execution [robertwb] [BEAM-1872] Add IdentityWindowFn for use in Reshuffle (#4040) [lcwik] Update the version of org.xolstice.maven.plugins to 0.5.1 to attempt to [ekirpichov] [BEAM-3060] add TFRecordIOIT [ekirpichov] add post-review updates [dariusz.aniszewski] Added AvroIOIT with static schema [github] statesampler will not be relative-imported in test [github] Remove redundant six dependency [lcwik] Fix duplicate test name. [lcwik] [BEAM-2957] increase desiredBundleSize in testSplit so that ES slices [lcwik] [BEAM-2957] Change assert in testSplit to allow up to 50% of empty [ehudm] Show diffs when isort complains about Python import order. [tgroh] Remove DockerContainer from beam_fn_api [dariusz.aniszewski] added support for passing extra mvn properties to pkb to be transferred [altay] Use the correct endpoint for cancel call [klk] Add @RequiresStableInput annotation [github] Renaming properties form IO target counter name. [arostami] Ensure temp file is closed before returning. [altay] Adding debug server to sdk worker to get threaddumps [tgroh] Port DataBufferingOutboundObserver [tgroh] Add a ReferenceRunner class [tgroh] Implement JobService#run in the ReferenceRunner [tgroh] Start Servers only in a ServerFactory [robertwb] [BEAM-3183] Add runner.run(transform) to Python SDK. [tgroh] Use FnDataReceiver in the SDK Harness [tgroh] Replace ThrowingConsumer with FnDataReceiver [lcwik] Replaces BoundedSource with a composite transform when using Fn API [github] Minor changes to the FnApi Runner to allow Java to run. (#4220) [github] Followup fix to pr/4195. [herohde] Add worker id to provisioning api [dariusz.aniszewski] reshuffle for AvroIOIT and TFRecordIOIT [lcwik] Fix javadoc in FnDataReceiver [mariagh] Add Python processing-time timers and clock injection [mariagh] Fix linter error [mariagh] Add review comments [mariagh] Address review comments [mariagh] borrar [mariagh] Add review commnets [dariusz.aniszewski] job for performance tests of file-based IOs [github] Allow Java Harness to understand Fns the Java Runner produces. (#4214) [altay] Add grpcio upper bound [aaltay] [BEAM-3189] Sdk worker multithreading (#4134) [aaltay] [BEAM-2101] Allow anonymous access of GCS blobs. (#4254) [dimon.ivan] Moves out common configuration parts from BigtableIO.Read and [jbonofre] [BEAM-1920] Upgrade to Spark runner to Spark 2.2.1 [jbonofre] [BEAM-3340] Update Flink Runner to Flink 1.4.0 [robertwb] [BEAM-3143] Type Inference Python 3 Compatibility (#4183) [robertwb] Fix getitem and list comprehension type inference. [lcwik] [BEAM-2929] Remove Dataflow expansions for PCollectionView that have [ekirpichov] [BEAM-2865] Introduces FileIO.write() and adds support for it in AvroIO [ekirpichov] Addresses comments [github] Guard against closing data channel twice (#4283) [dariusz.aniszewski] make it possible to trigger this job from PR [jbonofre] Fix global match test in localfilesystem [robertwb] [BEAM-3369] Fix typehint errors introduced in PR #4282. [robertwb] [BEAM-3369] Allow any coder to specify its deterministic variant. [chamikara] Adds support for processing Splittable DoFns using DirectRunner. [kirpichov] Makes gradle findbugs plugin produce an html report rather than xml [aaltay] [BEAM-3174] Don't parse the release version just directly use the [github] [BEAM-3357] Remove upper bound for protobuf (#4274) [kirpichov] Supports a sink for XmlIO [kirpichov] Adds TFRecordIO.sink() [github] [BEAM-3041] Support various pip based dependency options for python sdk [dariusz.aniszewski] FIX: remove overriding Google project in file-based IOs performance [chamikara] Set min version of protobof so that the range is compatible with grpc [altay] Update wordcount fnapi example to use now supported save_main_session [github] [BEAM-2937] Basic PGBK combiner lifting. (#4290) [github] Fix for side inputs following flatten/gbk. (#4281) [ehudm] Enable multiprocess docs generation [aaltay] [BEAM-3357] Add generated pb2 files to mvn clean. (#4287) [chamikara] Update BigTable client version [ehudm] Initial implementation of HadoopFileSsytem. [github] [BEAM-3060] explicitly use Apache's Google project for file-based [jbonofre] Remove semicolon [lcwik] Increment the Dataflow container version [lcwik] [BEAM-981] Add parameter allowing adding jars to spark context [lcwik] Post code review amendments and refactor [lcwik] Remove @JsonIgnore to allow setting filesToStage from command line [lcwik] Post code review amendments [lcwik] Revert "Update BigTable client version" [kirpichov] Splits BigQueryIOTest into a read and a write test [kirpichov] Some cleanups in write test [lcwik] Translate Flink batch nodes by URN [github] Remove extraneous logging. [kirpichov] Some cleanups in read test too [ankurgoenka] Clean data receive queue [github] Update dependency.py [dariusz.aniszewski] use dedicated BigQuery table for performance tests of FileBasedIOIT [dariusz.aniszewski] enable large-scale test for FileBasedIOIT [lcwik] Made replaceTransforms protected so that DataflowRunner subclass can [github] Use beam-master-$DATE pattern for default Dataflow containers in [altay] Upgrade oauth2cleitn and google-apitools dependencies [altay] Use pip download instead of the deprecated pip install --download [altay] Remove Queue based 60 seconds timeout for GCS io. [valentyn] Make release versions of Python sdks use containers tagged with the [mariagh] Address review comment [github] Update dependency.py [valentyn] Add unit tests to check default values of worker harness container [yiyan.lyy] Set utf-16 as SQL DSL default charset [github] fixup: lint [lcwik] Allow Python postcommits to run on beam{1,2,3}. [lcwik] Fixes a typo in type signature of [david.sabater] Added ProxyInfoFromEnvironmentVar() & GetNewHttp() methods to get proxy [david.sabater] Added parsing HTTP_PROXY variable in uppercase as well. [david.sabater] Some nit-picks around lower cases. [david.sabater] Adding example to show http_proxy format and accept only lower case [david.sabater] Added warnings about malformed proxy URL Removed initialising ProxyInfo [david.sabater] Reformatted function names from camel case to underscores. [david.sabater] Cleaned function get_new_http, removing unnecessary arguments, kwargs [david.sabater] Changed environment variable format checks using regexp (re.match) [david.sabater] Initialised proxy_info = None [david.sabater] Added extra newline before Returns section [david.sabater] Wrap to 80 chars for lint [david.sabater] Lint - Added two blank lines [david.sabater] Merged changes from gcsio [david.sabater] Shortening condition. [david.sabater] Reverting splitting comments longer than 80 characters to keep them in [david.sabater] Returning None and removing unnecessary if condition. [david.sabater] Correct os.environ.get call Fix Docstring conventions as per PEP 257 [david.sabater] Fixing trailing whitespace [iemejia] Disable incremental compilation for SQL module [iemejia] Remove repeated slf4j-api dependency from java-fn-execution module [lcwik] [SQL] Fix shading configuration [kedin] [Nexmark][SQL] Implement sql query 0 [lcwik] Fix typechecking for FluentIterable use. [lcwik] Avoid importing testing hierarchy for Javadoc comments. [lcwik] Build unshaded JARs. [github] Correctly handle pass-through "composites" in the Fn Api Runner. (#4326) [aromanenko-dev] [BEAM-3228] Add fix to prevent crashing forked VM during running Kinesis [iemejia] [BEAM-3404] Update KinesisIO to use AWS SDK 1.11.255 and KCL 1.8.8 [iemejia] [BEAM-3404] Fix and enable KinesisReaderIT [chamikara] Updates common.py due to a Cython bug. [robertwb] Plumb user metrics through Fn API. [mariagh] Clean mock import [iemejia] [BEAM-3187] Ensure that teardown is called in case of Exception on [iemejia] [BEAM-3187] Enable PardoLifecycleTest for the Spark runner [ankurgoenka] [BEAM-3411] Fixing KeyError when on progress update request [iemejia] [BEAM-3373] Add serviceEndpoint parameter to KinesisIO [chamikara] Reduce the size of file-based I/O tests [chamikara] Reduce the size of file-based I/O tests [jb] Exclude go from rat configuration, use .gitignore for excludes and align [tgroh] Add RemoteGrpcPort[Read/Write] abstractions [iemejia] [BEAM-3422] Update reproducible-build-maven-plugin to version 0.4 [robertwb] Return state from wait_until_finish. [ehudm] Cleanup DeprecationWarning for BaseException.message. [jbonofre] [BEAM-3282] MqttIO reader now use receive for timeout in order to return [arostami] Fixes unclosed files in VcfSource when exception is thrown. [kenn] Shade protobuf-java-util alongside protobuf-java [kenn] Remove Dataflow hack from shared translation code [lcwik] Add the ability to create an offline repository using [lcwik] [BEAM-2863] Create an EncodedBoundedWindow allowing runners to pass [swegner] Increment Dataflow worker container version [ryan] Enable multiple stopping conditions for KafkaIO ------------------------------------------ [...truncated 18.00 KB...] 2018-01-11 18:05:40,267 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:05:40,276 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:06:06,155 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:06:06,162 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:06:23,207 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:06:23,224 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:06:43,783 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:06:43,801 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:06:58,899 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:06:58,905 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:07:27,823 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:07:27,830 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:07:49,640 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:07:49,649 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:08:11,795 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:08:11,801 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:08:35,299 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:08:35,320 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:09:01,374 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:09:01,381 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:09:17,627 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:09:17,642 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:09:35,323 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:09:35,332 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:10:00,310 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:10:00,317 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:10:26,118 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:10:26,124 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:10:43,966 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:10:43,973 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:11:09,171 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:11:09,177 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:11:38,191 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:11:38,200 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:12:01,738 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:12:01,745 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:12:30,298 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:12:30,304 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:12:52,207 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:12:52,215 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:13:11,915 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:13:11,921 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:13:29,653 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:13:29,659 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:13:45,910 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:13:45,917 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:14:11,705 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:14:11,711 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:14:29,853 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:14:29,860 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:14:52,643 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:14:52,649 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:15:10,686 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:15:10,693 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:15:34,932 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:15:34,939 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:15:56,644 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:15:56,651 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:16:11,818 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:16:11,824 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:16:41,401 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:16:41,408 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:17:10,943 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:17:10,963 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:17:29,344 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:17:29,354 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:17:49,539 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:17:49,556 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:18:11,155 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:18:11,172 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:18:35,163 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:18:35,171 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:18:58,043 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:18:58,060 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:19:24,191 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:19:24,203 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:19:42,166 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:19:42,177 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:20:01,462 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:20:01,469 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:20:17,741 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:20:17,747 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:20:39,681 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:20:39,688 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:21:03,623 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:21:03,649 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:21:21,959 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:21:21,981 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:21:48,467 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:21:48,475 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:22:11,955 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:22:11,962 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:22:27,446 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:22:27,456 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:22:55,681 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:22:55,690 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:23:12,304 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:23:12,313 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:23:33,731 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:23:33,741 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:24:01,753 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:24:01,760 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:24:30,764 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:24:30,772 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:24:58,091 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f 2018-01-11 18:24:58,099 cc074d77 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark Traceback (most recent call last): File " line 597, in RunBenchmark DoPreparePhase(spec, detailed_timer) File " line 481, in DoPreparePhase spec.BenchmarkPrepare(spec) File " line 109, in Prepare kubernetes_helper.CreateAllFiles(getKubernetesScripts()) File " line 54, in CreateAllFiles CreateFromFile(file) File " line 37, in CreateFromFile vm_util.IssueRetryableCommand(create_cmd) File " line 247, in WrappedFunction return f(*args, **kwargs) File " line 361, in IssueRetryableCommand stdout, stderr, retcode = IssueCommand(cmd, env=env) File " line 298, in IssueCommand stderr=tf_err, cwd=cwd) File "/usr/lib/python2.7/subprocess.py", line 710, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory 2018-01-11 18:24:58,100 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark 2018-01-11 18:24:58,101 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:24:58,124 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:25:27,055 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:25:27,064 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:25:51,748 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:25:51,756 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:26:20,615 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:26:20,626 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:26:36,111 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:26:36,119 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:26:52,659 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:26:52,665 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:27:09,414 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:27:09,421 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:27:27,611 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:27:27,618 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:27:50,675 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:27:50,685 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:28:10,526 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:28:10,532 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:28:28,807 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:28:28,817 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:28:57,455 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:28:57,464 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:29:13,348 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:29:13,354 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:29:40,563 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:29:40,569 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:29:58,527 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:29:58,544 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:30:13,799 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:30:13,820 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:30:37,571 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:30:37,580 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:30:57,231 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:30:57,244 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:31:21,583 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:31:21,605 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:31:43,181 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:31:43,200 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:32:01,195 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:32:01,216 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:32:31,147 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:32:31,168 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:32:51,973 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:32:51,986 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:33:11,407 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:33:11,416 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:33:30,735 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:33:30,756 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:33:50,339 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:33:50,352 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:34:16,470 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:34:16,478 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:34:38,650 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:34:38,669 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:34:53,897 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:34:53,906 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:35:14,150 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:35:14,159 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:35:41,906 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:35:41,913 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:36:01,452 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:36:01,458 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:36:18,621 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:36:18,629 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:36:47,283 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:36:47,288 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:37:12,634 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:37:12,639 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:37:34,959 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:37:34,965 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:37:58,007 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:37:58,013 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:38:20,224 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:38:20,235 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:38:37,727 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:38:37,734 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:38:53,315 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:38:53,321 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:39:16,647 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:39:16,668 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:39:38,967 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:39:38,975 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:40:08,583 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:40:08,593 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:40:23,853 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:40:23,859 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:40:46,279 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:40:46,292 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:41:12,219 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:41:12,235 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:41:39,651 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:41:39,659 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:42:04,251 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:42:04,260 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:42:23,340 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:42:23,347 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:42:43,895 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:42:43,903 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:43:10,863 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:43:10,870 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:43:35,391 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:43:35,397 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:44:00,140 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:44:00,147 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:44:18,162 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:44:18,168 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Retrying exception running IssueRetryableCommand: [Errno 2] No such file or directory 2018-01-11 18:44:43,288 cc074d77 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f 2018-01-11 18:44:43,295 cc074d77 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue. 2018-01-11 18:44:43,354 cc074d77 MainThread INFO Benchmark run statuses: --------------------------------------------------------------- Name UID Status --------------------------------------------------------------- beam_integration_benchmark beam_integration_benchmark0 FAILED --------------------------------------------------------------- Success rate: 0.00% (0/1) 2018-01-11 18:44:43,354 cc074d77 MainThread INFO Complete logs can be found at: /tmp/perfkitbenchmarker/runs/cc074d77/pkb.log 2018-01-11 18:44:43,355 cc074d77 MainThread INFO Completion statuses can be found at: /tmp/perfkitbenchmarker/runs/cc074d77/completion_statuses.json Build step 'Execute shell' marked build as failure