Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B217A200C5B for ; Thu, 27 Apr 2017 11:09:32 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id B0933160BA7; Thu, 27 Apr 2017 09:09:32 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 34F7A160B98 for ; Thu, 27 Apr 2017 11:09:30 +0200 (CEST) Received: (qmail 2422 invoked by uid 500); 27 Apr 2017 09:09:29 -0000 Mailing-List: contact commits-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.apache.org Delivered-To: mailing list commits@beam.apache.org Received: (qmail 2413 invoked by uid 99); 27 Apr 2017 09:09:29 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Apr 2017 09:09:29 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id C4C78C074C for ; Thu, 27 Apr 2017 09:09:28 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.201 X-Spam-Level: X-Spam-Status: No, score=-99.201 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id ij3hTh3mkwpg for ; Thu, 27 Apr 2017 09:09:06 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 97D505F4EE for ; Thu, 27 Apr 2017 09:09:05 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 6CF9BE04AB for ; Thu, 27 Apr 2017 09:09:04 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 1C64C21DDE for ; Thu, 27 Apr 2017 09:09:04 +0000 (UTC) Date: Thu, 27 Apr 2017 09:09:04 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: commits@beam.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (BEAM-1981) Serialization error with TimerInternals in ApexGroupByKeyOperator MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 27 Apr 2017 09:09:32 -0000 [ https://issues.apache.org/jira/browse/BEAM-1981?page=3Dcom.atlassian.= jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D15986= 244#comment-15986244 ]=20 ASF GitHub Bot commented on BEAM-1981: -------------------------------------- GitHub user tweise opened a pull request: https://github.com/apache/beam/pull/2734 [BEAM-1981] Fix ApexTimerInternals serialization error. Be sure to do all of the following to help us incorporate your contribu= tion quickly and easily: =20 - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enabl= e Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.or= g/licenses/icla.pdf). =20 --- R: @kennknowles @dhalperi=20 You can merge this pull request into a Git repository by running: $ git pull https://github.com/tweise/beam BEAM-1981-ApexTimerInternals Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2734.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2734 =20 ---- commit d85da488970677c5b90eb5dd7f2fbb88fb68ec09 Author: Thomas Weise Date: 2017-04-27T09:06:50Z BEAM-1981 Fix ApexTimerInternals serialization error. ---- > Serialization error with TimerInternals in ApexGroupByKeyOperator > ----------------------------------------------------------------- > > Key: BEAM-1981 > URL: https://issues.apache.org/jira/browse/BEAM-1981 > Project: Beam > Issue Type: Bug > Components: runner-apex > Reporter: Daniel Halperin > Assignee: Thomas Weise > Fix For: First stable release > > > Logs below. We tried switching to Java serialization, but that didn't wor= k. We made the field transient (which is broken but let us make progress) a= nd that did. > Stack trace > {code} > com.esotericsoftware.kryo.KryoException: Class cannot be created (missing= no-arg constructor): org.apache.beam.runners.core.AutoValue_TimerInternals= _TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > context: PTContainer[id=3D6(container_1492195730173_0001_01_000012),stat= e=3DACTIVE,operators=3D[ > {code} > Larger logs with more scope: > {code}=09 > 2017-04-14 18:56:49,961 INFO com.datatorrent.stram.StreamingAppMaster: Ma= ster starting with classpath: ./portability-demo-bundled-apex.jar:/etc/hado= op/conf:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-annotations.= jar:/usr/lib/hadoop/hadoop-auth-2.7.3.jar:/usr/lib/hadoop/hadoop-nfs.jar:/u= sr/lib/hadoop/hadoop-common-2.7.3-tests.jar:/usr/lib/hadoop/hadoop-annotati= ons-2.7.3.jar:/usr/lib/hadoop/hadoop-nfs-2.7.3.jar:/usr/lib/hadoop/hadoop-c= ommon.jar:/usr/lib/hadoop/hadoop-common-2.7.3.jar:/usr/lib/hadoop/lib/commo= ns-configuration-1.6.jar:/usr/lib/hadoop/lib/snappy-java-1.0.5.jar:/usr/lib= /hadoop/lib/curator-recipes-2.7.1.jar:/usr/lib/hadoop/lib/commons-lang-2.6.= jar:/usr/lib/hadoop/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop/lib/hadoop-lz= o-0.4.19.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/had= oop/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.9.13.= jar:/usr/lib/hadoop/lib/jersey-core-1.9.jar:/usr/lib/hadoop/lib/netty-3.6.2= .Final.jar:/usr/lib/hadoop/lib/bigquery-connector-0.10.1-hadoop2.jar:/usr/l= ib/hadoop/lib/slf4j-api-1.7.10.jar:/usr/lib/hadoop/lib/avro-1.7.7.jar:/usr/= lib/hadoop/lib/stax-api-1.0-2.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/= usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/us= r/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/curator-framewor= k-2.7.1.jar:/usr/lib/hadoop/lib/api-util-1.0.0-M20.jar:/usr/lib/hadoop/lib/= slf4j-log4j12-1.7.10.jar:/usr/lib/hadoop/lib/commons-io-2.4.jar:/usr/lib/ha= doop/lib/gcs-connector-1.6.0-hadoop2.jar:/usr/lib/hadoop/lib/commons-digest= er-1.8.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/zo= okeeper-3.4.6.jar:/usr/lib/hadoop/lib/jets3t-0.9.0.jar:/usr/lib/hadoop/lib/= jetty-util-6.1.26.jar:/usr/lib/hadoop/lib/curator-client-2.7.1.jar:/usr/lib= /hadoop/lib/htrace-core-3.1.0-incubating.jar:/usr/lib/hadoop/lib/protobuf-j= ava-2.5.0.jar:/usr/lib/hadoop/lib/jetty-6.1.26.jar:/usr/lib/hadoop/lib/comm= ons-httpclient-3.1.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop= /lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/li= b/jsr305-3.0.0.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib= /activation-1.1.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/u= sr/lib/hadoop/lib/httpclient-4.2.5.jar:/usr/lib/hadoop/lib/jackson-core-asl= -1.9.13.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/jer= sey-server-1.9.jar:/usr/lib/hadoop/lib/commons-collections-3.2.2.jar:/usr/l= ib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/hadoop/lib/jsp= -api-2.1.jar:/usr/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/usr/lib/hadoop/li= b/gson-2.2.4.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/h= adoop/lib/jersey-json-1.9.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hado= op/lib/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop/lib/httpcore-4.2.5.jar:/u= sr/lib/hadoop/lib/junit-4.11.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/us= r/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/apacheds-i18n-2.0.= 0-M15.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/jackson-= xc-1.9.13.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/com= mons-compress-1.4.1.jar:/usr/lib/hadoop/lib/commons-math3-3.1.1.jar:/usr/li= b/hadoop-hdfs/hadoop-hdfs-2.7.3.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs.ja= r:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.7= .3-tests.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.7.3.jar:/usr/lib/hadoop= -hdfs/lib/xml-apis-1.3.04.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.6.jar= :/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-hdf= s/lib/commons-logging-1.1.3.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.9.ja= r:/usr/lib/hadoop-hdfs/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-hdfs/lib/l= eveldbjni-all-1.8.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib= /hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/commons-io-= 2.4.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.jar:/usr/lib/hadoop-hdfs= /lib/htrace-core-3.1.0-incubating.jar:/usr/lib/hadoop-hdfs/lib/protobuf-jav= a-2.5.0.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.jar:/usr/lib/hadoop-hdfs/= lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hado= op-hdfs/lib/xercesImpl-2.9.1.jar:/usr/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:= /usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-hdfs/l= ib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/usr/= lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/usr/lib/hadoop-hdfs/lib/nett= y-all-4.0.23.Final.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop= -hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/commons-configuration-= 1.6.jar:/usr/lib/hadoop-mapreduce/hadoop-auth.jar:/usr/lib/hadoop-mapreduce= /snappy-java-1.0.5.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jo= bclient.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/li= b/hadoop-mapreduce/curator-recipes-2.7.1.jar:/usr/lib/hadoop-mapreduce/comm= ons-lang-2.6.jar:/usr/lib/hadoop-mapreduce/hamcrest-core-1.3.jar:/usr/lib/h= adoop-mapreduce/hadoop-auth-2.7.3.jar:/usr/lib/hadoop-mapreduce/jackson-map= per-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/commons-logging-1.1.3.jar:/usr= /lib/hadoop-mapreduce/aws-java-sdk-1.7.4.jar:/usr/lib/hadoop-mapreduce/jack= son-jaxrs-1.9.13.jar:/usr/lib/hadoop-mapreduce/jersey-core-1.9.jar:/usr/lib= /hadoop-mapreduce/hadoop-archives-2.7.3.jar:/usr/lib/hadoop-mapreduce/netty= -3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common-2= .7.3.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-common.jar:/usr/= lib/hadoop-mapreduce/hadoop-extras-2.7.3.jar:/usr/lib/hadoop-mapreduce/hado= op-sls-2.7.3.jar:/usr/lib/hadoop-mapreduce/hadoop-aws-2.7.3.jar:/usr/lib/ha= doop-mapreduce/avro-1.7.7.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-cl= ient-core-2.7.3.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.7= .3.jar:/usr/lib/hadoop-mapreduce/stax-api-1.0-2.jar:/usr/lib/hadoop-mapredu= ce/hadoop-sls.jar:/usr/lib/hadoop-mapreduce/servlet-api-2.5.jar:/usr/lib/ha= doop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/hadoop-mapred= uce-client-jobclient-2.7.3.jar:/usr/lib/hadoop-mapreduce/mockito-all-1.8.5.= jar:/usr/lib/hadoop-mapreduce/xz-1.0.jar:/usr/lib/hadoop-mapreduce/commons-= codec-1.4.jar:/usr/lib/hadoop-mapreduce/curator-framework-2.7.1.jar:/usr/li= b/hadoop-mapreduce/api-util-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/hadoop-= azure-2.7.3.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-shuffle-2= .7.3.jar:/usr/lib/hadoop-mapreduce/commons-io-2.4.jar:/usr/lib/hadoop-mapre= duce/hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/commons-= digester-1.8.jar:/usr/lib/hadoop-mapreduce/hadoop-distcp-2.7.3.jar:/usr/lib= /hadoop-mapreduce/hadoop-archives.jar:/usr/lib/hadoop-mapreduce/jaxb-impl-2= .2.3-1.jar:/usr/lib/hadoop-mapreduce/zookeeper-3.4.6.jar:/usr/lib/hadoop-ma= preduce/jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/jetty-util-6.1.26.jar:/u= sr/lib/hadoop-mapreduce/curator-client-2.7.1.jar:/usr/lib/hadoop-mapreduce/= htrace-core-3.1.0-incubating.jar:/usr/lib/hadoop-mapreduce/protobuf-java-2.= 5.0.jar:/usr/lib/hadoop-mapreduce/joda-time-2.9.9.jar:/usr/lib/hadoop-mapre= duce/jetty-6.1.26.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/h= adoop-mapreduce/commons-httpclient-3.1.jar:/usr/lib/hadoop-mapreduce/log4j-= 1.2.17.jar:/usr/lib/hadoop-mapreduce/jsch-0.1.42.jar:/usr/lib/hadoop-mapred= uce/hadoop-gridmix-2.7.3.jar:/usr/lib/hadoop-mapreduce/xmlenc-0.52.jar:/usr= /lib/hadoop-mapreduce/jsr305-3.0.0.jar:/usr/lib/hadoop-mapreduce/hadoop-ext= ras.jar:/usr/lib/hadoop-mapreduce/azure-storage-2.0.0.jar:/usr/lib/hadoop-m= apreduce/jettison-1.1.jar:/usr/lib/hadoop-mapreduce/activation-1.1.jar:/usr= /lib/hadoop-mapreduce/hadoop-openstack-2.7.3.jar:/usr/lib/hadoop-mapreduce/= hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin-= 2.7.3.jar:/usr/lib/hadoop-mapreduce/hadoop-datajoin.jar:/usr/lib/hadoop-map= reduce/hadoop-streaming-2.7.3.jar:/usr/lib/hadoop-mapreduce/commons-beanuti= ls-core-1.8.0.jar:/usr/lib/hadoop-mapreduce/httpclient-4.2.5.jar:/usr/lib/h= adoop-mapreduce/hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/ja= ckson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/commons-lang3-3.3.2.jar= :/usr/lib/hadoop-mapreduce/commons-cli-1.2.jar:/usr/lib/hadoop-mapreduce/je= rsey-server-1.9.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobcl= ient-2.7.3-tests.jar:/usr/lib/hadoop-mapreduce/commons-collections-3.2.2.ja= r:/usr/lib/hadoop-mapreduce/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/lib/= hadoop-mapreduce/jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/java-xmlbuilder-= 0.4.jar:/usr/lib/hadoop-mapreduce/jackson-annotations-2.2.3.jar:/usr/lib/ha= doop-mapreduce/gson-2.2.4.jar:/usr/lib/hadoop-mapreduce/metrics-core-3.0.1.= jar:/usr/lib/hadoop-mapreduce/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-m= apreduce/hadoop-mapreduce-client-hs-plugins-2.7.3.jar:/usr/lib/hadoop-mapre= duce/hadoop-mapreduce-client-hs-2.7.3.jar:/usr/lib/hadoop-mapreduce/hadoop-= mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/jackson-databind-2.2.3.j= ar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-hs-plugins.jar:/usr/li= b/hadoop-mapreduce/hadoop-mapreduce-client-app-2.7.3.jar:/usr/lib/hadoop-ma= preduce/jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/asm-3.2.jar:/usr/lib/= hadoop-mapreduce/api-asn1-api-1.0.0-M20.jar:/usr/lib/hadoop-mapreduce/httpc= ore-4.2.5.jar:/usr/lib/hadoop-mapreduce/hadoop-openstack.jar:/usr/lib/hadoo= p-mapreduce/hadoop-rumen-2.7.3.jar:/usr/lib/hadoop-mapreduce/hadoop-ant-2.7= .3.jar:/usr/lib/hadoop-mapreduce/junit-4.11.jar:/usr/lib/hadoop-mapreduce/h= adoop-aws.jar:/usr/lib/hadoop-mapreduce/jaxb-api-2.2.2.jar:/usr/lib/hadoop-= mapreduce/commons-net-3.1.jar:/usr/lib/hadoop-mapreduce/apacheds-i18n-2.0.0= -M15.jar:/usr/lib/hadoop-mapreduce/jackson-core-2.2.3.jar:/usr/lib/hadoop-m= apreduce/guava-11.0.2.jar:/usr/lib/hadoop-mapreduce/jackson-xc-1.9.13.jar:/= usr/lib/hadoop-mapreduce/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/hadoop= -azure.jar:/usr/lib/hadoop-mapreduce/commons-compress-1.4.1.jar:/usr/lib/ha= doop-mapreduce/hadoop-ant.jar:/usr/lib/hadoop-mapreduce/hadoop-gridmix.jar:= /usr/lib/hadoop-mapreduce/hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/commo= ns-math3-3.1.1.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.5.jar:/usr= /lib/hadoop-mapreduce/lib/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/l= ib/jackson-mapper-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/guice-servle= t-3.0.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/usr/lib/hadoop= -mapreduce/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop-mapreduce/lib/leveldbj= ni-all-1.8.jar:/usr/lib/hadoop-mapreduce/lib/avro-1.7.7.jar:/usr/lib/hadoop= -mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar= :/usr/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/usr/lib/hadoop-mapreduce= /lib/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/u= sr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/= jackson-core-asl-1.9.13.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.9= .jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapre= duce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/junit-4.11.jar:/usr/lib/= hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/common= s-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/us= r/lib/hadoop-yarn/hadoop-yarn-api-2.7.3.jar:/usr/lib/hadoop-yarn/hadoop-yar= n-server-resourcemanager-2.7.3.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applica= tions-unmanaged-am-launcher-2.7.3.jar:/usr/lib/hadoop-yarn/hadoop-yarn-clie= nt-2.7.3.jar:/usr/lib/hadoop-yarn/hadoop-yarn-common-2.7.3.jar:/usr/lib/had= oop-yarn/hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/hadoop-yar= n-server-web-proxy-2.7.3.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-resour= cemanager.jar:/usr/lib/hadoop-yarn/hadoop-yarn-registry.jar:/usr/lib/hadoop= -yarn/hadoop-yarn-registry-2.7.3.jar:/usr/lib/hadoop-yarn/hadoop-yarn-appli= cations-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:= /usr/lib/hadoop-yarn/hadoop-yarn-server-sharedcachemanager.jar:/usr/lib/had= oop-yarn/hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn= /hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-c= ommon.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice= -2.7.3.jar:/usr/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2= .7.3.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests-2.7.3.jar:/usr/lib/h= adoop-yarn/hadoop-yarn-server-nodemanager-2.7.3.jar:/usr/lib/hadoop-yarn/ha= doop-yarn-server-sharedcachemanager-2.7.3.jar:/usr/lib/hadoop-yarn/hadoop-y= arn-server-applicationhistoryservice.jar:/usr/lib/hadoop-yarn/hadoop-yarn-s= erver-common-2.7.3.jar:/usr/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/u= sr/lib/hadoop-yarn/hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/hadoop-yarn-= common.jar:/usr/lib/hadoop-yarn/lib/*' > 2017-04-14 18:56:49,966 INFO com.datatorrent.stram.StreamingAppMaster: ve= rsion: 3.5.0 from rev: 6de8828 branch: 6de8828e4f3d5734d0a6f9c1be0aa7057cb6= 0ac8 by Thomas Weise on 06.12.2016 @ 22:11:51 PST > 2017-04-14 18:56:49,967 INFO com.datatorrent.stram.StreamingAppMaster: ap= pmaster env: > PATH=3D/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games > HADOOP_CONF_DIR=3D/etc/hadoop/conf > MAX_APP_ATTEMPTS=3D2 > DATAPROC_MASTER_STANDALONE_COMPONENTS=3Dhadoop-hdfs-secondarynamenode > DATAPROC_COMMON_COMPONENTS=3Dopenjdk-8-jdk libjansi-java python-numpy lib= mysql-java hadoop-client hive pig spark-core spark-python spark-r autofs nf= s-common libhdfs0 libsnappy1 libatlas3-base libopenblas-base libapr1 vim gi= t bash-completion spark-yarn-shuffle spark-datanucleus spark-extras hadoop-= lzo > MAIL=3D/var/mail/yarn > LD_LIBRARY_PATH=3D:/usr/lib/hadoop/lib/native:/usr/lib/hadoop/lib/native > HADOOP_HOME_WARN_SUPPRESS=3Dtrue > DATAPROC_WORKER_COMPONENTS=3Dhadoop-hdfs-datanode hadoop-yarn-nodemanager > LOGNAME=3Ddhalperi > JVM_PID=3D2123 > JSVC_HOME=3D/usr/lib/bigtop-utils > PWD=3D/hadoop/yarn/nm-local-dir/usercache/dhalperi/appcache/application_1= 492195730173_0001/container_1492195730173_0001_01_000001 > HADOOP_YARN_USER=3Dyarn > HADOOP_PREFIX=3D/usr/lib/hadoop > LOCAL_DIRS=3D/hadoop/yarn/nm-local-dir/usercache/dhalperi/appcache/applic= ation_1492195730173_0001 > YARN_IDENT_STRING=3Dyarn > SHELL=3D/bin/bash > ALPN_JAR=3D/usr/local/share/google/alpn/alpn-boot-8.1.7.v20160121.jar > YARN_CONF_DIR=3D/etc/hadoop/conf > _SYSTEMCTL_SKIP_REDIRECT=3Dtrue > LOG_DIRS=3D/var/log/hadoop-yarn/userlogs/application_1492195730173_0001/c= ontainer_1492195730173_0001_01_000001 > NM_AUX_SERVICE_mapreduce_shuffle=3DAAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= AAAAA=3D > BDUTIL_DIR=3D/usr/local/share/google/dataproc/bdutil-dataproc-20170410-12= 4434-RC1 > HADOOP_YARN_HOME=3D/usr/lib/hadoop-yarn > YARN_PID_DIR=3D/var/run/hadoop-yarn > DATAPROC_MASTER_EXCLUSIVE_COMPONENTS=3Dhadoop-mapreduce-historyserver hiv= e-metastore hive-server2 nfs-kernel-server spark-history-server > NM_AUX_SERVICE_spark_shuffle=3D > HADOOP_HOME=3D/usr/lib/hadoop > SHLVL=3D5 > YARN_ROOT_LOGGER=3DINFO,RFA > JAVA_HOME=3D/usr/lib/jvm/java-8-openjdk-amd64 > LANG=3Den_US.UTF-8 > APP_SUBMIT_TIME_ENV=3D1492196207488 > NM_HOST=3Dgaming-apex-dhalperi-w-15.c.clouddfe.google.com.internal > YARN_LOGFILE=3Dyarn-yarn-nodemanager-gaming-apex-dhalperi-w-15.log > HADOOP_USER_NAME=3Ddhalperi > DATAPROC_AGENT_JAR=3D/usr/local/share/google/dataproc/agent-20170410-1244= 34-RC1.jar > HADOOP_LIBEXEC_DIR=3D/usr/lib/hadoop/libexec > YARN_LOG_DIR=3D/var/log/hadoop-yarn > HADOOP_HDFS_HOME=3D/usr/lib/hadoop-hdfs > HADOOP_MAPRED_HOME=3D/usr/lib/hadoop-mapreduce > HADOOP_COMMON_HOME=3D/usr/lib/hadoop > _=3D/usr/lib/jvm/java-8-openjdk-amd64/bin/java > APPLICATION_WEB_PROXY_BASE=3D/proxy/application_1492195730173_0001 > NM_HTTP_PORT=3D8042 > NM_PORT=3D53054 > USER=3Ddhalperi > CLASSPATH=3D./*:/etc/hadoop/conf:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*:= /usr/lib/hadoop-hdfs/*:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-mapreduce= /*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-yarn/*:/usr/lib/hadoop-y= arn/lib/*' > DATAPROC_MASTER_HA_COMPONENTS=3Dhadoop-hdfs-journalnode hadoop-hdfs-zkfc = zookeeper-server > HADOOP_TOKEN_FILE_LOCATION=3D/hadoop/yarn/nm-local-dir/usercache/dhalperi= /appcache/application_1492195730173_0001/container_1492195730173_0001_01_00= 0001/container_tokens > DATAPROC_MASTER_COMPONENTS=3Dhadoop-hdfs-namenode hadoop-yarn-resourceman= ager mysql-server > YARN_NICENESS=3D0 > HOME=3D/home/ > CONTAINER_ID=3Dcontainer_1492195730173_0001_01_000001 > MALLOC_ARENA_MAX=3D4 > 2017-04-14 18:56:50,236 INFO com.datatorrent.stram.StreamingAppMaster: In= itializing Application Master. > 2017-04-14 18:56:50,311 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Application master, appId=3D1, clustertimestamp=3D1492195730173, attem= ptId=3D1 > 2017-04-14 18:56:52,099 INFO com.google.cloud.hadoop.fs.gcs.GoogleHadoopF= ileSystemBase: GHFS version: 1.6.0-hadoop2 > 2017-04-14 18:56:52,491 INFO com.datatorrent.common.util.AsyncFSStorageAg= ent: using /hadoop/yarn/nm-local-dir/usercache/dhalperi/appcache/applicatio= n_1492195730173_0001/container_1492195730173_0001_01_000001/tmp/chkp5833413= 517908828871 as the basepath for checkpointing. > 2017-04-14 18:56:56,041 INFO com.datatorrent.stram.FSRecoveryHandler: Cre= ating hdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/applicat= ion_1492195730173_0001/recovery/log > 2017-04-14 18:56:56,172 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Starting application with 10 operators in 10 containers > 2017-04-14 18:56:56,182 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Upper bound of the thread pool size is 500 > 2017-04-14 18:56:56,183 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : = 0 > 2017-04-14 18:56:56,201 INFO org.apache.hadoop.yarn.client.RMProxy: Conne= cting to ResourceManager at gaming-apex-dhalperi-m/10.240.169.55:8030 > 2017-04-14 18:56:56,224 INFO com.datatorrent.stram.StreamingContainerPare= nt: Config: Configuration: core-default.xml, core-site.xml, yarn-default.xm= l, yarn-site.xml, mapred-default.xml, mapred-site.xml, hdfs-default.xml, hd= fs-site.xml > 2017-04-14 18:56:56,224 INFO com.datatorrent.stram.StreamingContainerPare= nt: Listener thread count 30 > 2017-04-14 18:56:56,230 INFO org.apache.hadoop.ipc.CallQueueManager: Usin= g callQueue class java.util.concurrent.LinkedBlockingQueue > 2017-04-14 18:56:56,235 INFO org.apache.hadoop.ipc.Server: Starting Socke= t Reader #1 for port 57363 > 2017-04-14 18:56:56,256 INFO org.apache.hadoop.ipc.Server: IPC Server Res= ponder: starting > 2017-04-14 18:56:56,257 INFO org.apache.hadoop.ipc.Server: IPC Server lis= tener on 57363: starting > 2017-04-14 18:56:56,289 INFO com.datatorrent.stram.StreamingContainerPare= nt: Container callback server listening at gaming-apex-dhalperi-w-15/10.240= .169.52:57363 > 2017-04-14 18:56:56,334 INFO org.mortbay.log: Logging to org.slf4j.impl.L= og4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog > 2017-04-14 18:56:56,417 INFO org.apache.hadoop.security.authentication.se= rver.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, f= alling back to use random secrets. > 2017-04-14 18:56:56,424 INFO org.apache.hadoop.http.HttpRequestLog: Http = request log for http.requests.stram is not defined > 2017-04-14 18:56:56,432 INFO org.apache.hadoop.http.HttpServer2: Added gl= obal filter 'safety' (class=3Dorg.apache.hadoop.http.HttpServer2$QuotingInp= utFilter) > 2017-04-14 18:56:56,434 INFO org.apache.hadoop.http.HttpServer2: Added fi= lter static_user_filter (class=3Dorg.apache.hadoop.http.lib.StaticUserWebFi= lter$StaticUserFilter) to context stram > 2017-04-14 18:56:56,434 INFO org.apache.hadoop.http.HttpServer2: Added fi= lter static_user_filter (class=3Dorg.apache.hadoop.http.lib.StaticUserWebFi= lter$StaticUserFilter) to context static > 2017-04-14 18:56:56,435 INFO org.apache.hadoop.http.HttpServer2: Added fi= lter static_user_filter (class=3Dorg.apache.hadoop.http.lib.StaticUserWebFi= lter$StaticUserFilter) to context logs > 2017-04-14 18:56:56,437 INFO org.apache.hadoop.http.HttpServer2: adding p= ath spec: /stram/* > 2017-04-14 18:56:56,437 INFO org.apache.hadoop.http.HttpServer2: adding p= ath spec: /ws/* > 2017-04-14 18:56:56,658 INFO org.apache.hadoop.yarn.webapp.WebApps: Regis= tered webapp guice modules > 2017-04-14 18:56:56,659 INFO org.apache.hadoop.http.HttpServer2: Jetty bo= und to port 32869 > 2017-04-14 18:57:08,363 WARN com.datatorrent.stram.webapp.TypeGraphFactor= y: The size of precomputed type graph is 8136 KB > 2017-04-14 18:57:08,838 INFO org.apache.hadoop.yarn.webapp.WebApps: Web a= pp stram started at 32869 > 2017-04-14 18:57:08,838 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Started web service at port: 32869 > 2017-04-14 18:57:08,839 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Setting tracking URL to: gaming-apex-dhalperi-w-15:32869 > 2017-04-14 18:57:08,849 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Starting ApplicationMaster > 2017-04-14 18:57:08,850 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: number of tokens: 1 > 2017-04-14 18:57:08,908 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Max mem 12288m, Min mem 1024m, Max vcores 32000 and Min vcores 1 capab= ililty of resources in this cluster=20 > 2017-04-14 18:57:08,909 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Blacklist removal time in millis =3D 3600000, max consecutive node fai= lure count =3D 2147483647 > 2017-04-14 18:57:08,913 INFO org.apache.hadoop.yarn.client.RMProxy: Conne= cting to ResourceManager at gaming-apex-dhalperi-m/10.240.169.55:8032 > 2017-04-14 18:57:10,127 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 9,name=3DSumTeamScores/GroupByKey] > 2017-04-14 18:57:10,127 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 18:57:10,131 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 1,name=3DTextIO.Read/Read] > 2017-04-14 18:57:10,131 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 18:57:10,131 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 4,name=3DFixedWindows/Window.Assign] > 2017-04-14 18:57:10,131 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 18:57:10,131 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 10,name=3DSumTeamScores/ParDo(WriteWindowedFiles)/ParMultiDo(WriteWindowedF= iles)] > 2017-04-14 18:57:10,131 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 18:57:10,131 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 2,name=3DParseGameEvent/ParMultiDo(ParseEvent)] > 2017-04-14 18:57:10,131 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 18:57:10,131 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 8,name=3DSumTeamScores/ParDo(KeyByWindow)/ParMultiDo(KeyByWindow)] > 2017-04-14 18:57:10,131 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 18:57:10,131 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 6,name=3DSumTeamScores/Combine.perKey(SumInteger)/GroupByKey] > 2017-04-14 18:57:10,131 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 18:57:10,132 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 5,name=3DSumTeamScores/ParDo(KeyScoreByTeam)/ParMultiDo(KeyScoreByTeam)] > 2017-04-14 18:57:10,132 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 18:57:10,132 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 7,name=3DSumTeamScores/Combine.perKey(SumInteger)/Combine.GroupedValues/Par= Do(Anonymous)/ParMultiDo(Anonymous)] > 2017-04-14 18:57:10,132 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 18:57:10,132 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 3,name=3DSetTimestamps/ParMultiDo(SetTimestamps)] > 2017-04-14 18:57:10,132 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 18:57:10,132 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Asking RM for containers: [Capability[]Priority= [0], Capability[]Priority[1], Capability[]Priority[2], Capability[]Priority[3], C= apability[]Priority[4], Capability[]Priority[5], Capability[]Priority[6], Capabil= ity[]Priority[7], Capability[= ]Priority[8], Capability[]Priority[9]] > 2017-04-14 18:57:10,132 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[0] on= host: [null] > 2017-04-14 18:57:10,137 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[1] on= host: [null] > 2017-04-14 18:57:10,137 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[2] on= host: [null] > 2017-04-14 18:57:10,138 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[3] on= host: [null] > 2017-04-14 18:57:10,138 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[4] on= host: [null] > 2017-04-14 18:57:10,138 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[5] on= host: [null] > 2017-04-14 18:57:10,138 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[6] on= host: [null] > 2017-04-14 18:57:10,138 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[7] on= host: [null] > 2017-04-14 18:57:10,138 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[8] on= host: [null] > 2017-04-14 18:57:10,138 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[9] on= host: [null] > 2017-04-14 18:57:11,216 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-9.c.clouddfe.goo= gle.com.internal:37663 > 2017-04-14 18:57:11,216 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-11.c.clouddfe.go= ogle.com.internal:50931 > 2017-04-14 18:57:11,216 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-5.c.clouddfe.goo= gle.com.internal:33276 > 2017-04-14 18:57:11,216 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-1.c.clouddfe.goo= gle.com.internal:37140 > 2017-04-14 18:57:11,216 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-16.c.clouddfe.go= ogle.com.internal:52928 > 2017-04-14 18:57:11,216 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-12.c.clouddfe.go= ogle.com.internal:55762 > 2017-04-14 18:57:11,216 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-3.c.clouddfe.goo= gle.com.internal:44743 > 2017-04-14 18:57:11,216 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-2.c.clouddfe.goo= gle.com.internal:46370 > 2017-04-14 18:57:11,217 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-15.c.clouddfe.go= ogle.com.internal:53054 > 2017-04-14 18:57:11,217 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-18.c.clouddfe.go= ogle.com.internal:39804 > 2017-04-14 18:57:11,218 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 02, containerNode=3Dgaming-apex-dhalperi-w-9.c.clouddfe.google.com.internal= :37663, containerNodeURI=3Dgaming-apex-dhalperi-w-9.c.clouddfe.google.com.i= nternal:8042, containerResourceMemory2048, priority0 > 2017-04-14 18:57:11,245 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000002 > 2017-04-14 18:57:11,251 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 18:57:11,307 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 18:57:11,307 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000002 > 2017-04-14 18:57:11,307 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-9.c.clouddfe.google.com.intern= al:37663 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATIO= N_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appli= cation_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer_= 1492195730173_0001_01_000002 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.d= ir=3D com.datatorrent.stram.engine.StreamingContainer 1>/= stdout 2>/stderr =20 > 2017-04-14 18:57:11,317 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 03, containerNode=3Dgaming-apex-dhalperi-w-11.c.clouddfe.google.com.interna= l:50931, containerNodeURI=3Dgaming-apex-dhalperi-w-11.c.clouddfe.google.com= .internal:8042, containerResourceMemory2048, priority1 > 2017-04-14 18:57:11,321 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000002 > 2017-04-14 18:57:11,325 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000003 > 2017-04-14 18:57:11,326 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 18:57:11,332 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 18:57:11,332 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000003 > 2017-04-14 18:57:11,334 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-11.c.clouddfe.google.com.inter= nal:50931 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATI= ON_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appl= ication_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer= _1492195730173_0001_01_000003 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.= dir=3D com.datatorrent.stram.engine.StreamingContainer 1>= /stdout 2>/stderr =20 > 2017-04-14 18:57:11,334 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-9.c.clo= uddfe.google.com.internal:37663 > 2017-04-14 18:57:11,339 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 04, containerNode=3Dgaming-apex-dhalperi-w-5.c.clouddfe.google.com.internal= :33276, containerNodeURI=3Dgaming-apex-dhalperi-w-5.c.clouddfe.google.com.i= nternal:8042, containerResourceMemory2048, priority2 > 2017-04-14 18:57:11,342 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000003 > 2017-04-14 18:57:11,345 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000004 > 2017-04-14 18:57:11,346 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 18:57:11,350 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 18:57:11,351 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000004 > 2017-04-14 18:57:11,351 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-5.c.clouddfe.google.com.intern= al:33276 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATIO= N_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appli= cation_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer_= 1492195730173_0001_01_000004 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.d= ir=3D com.datatorrent.stram.engine.StreamingContainer 1>/= stdout 2>/stderr =20 > 2017-04-14 18:57:11,352 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 05, containerNode=3Dgaming-apex-dhalperi-w-1.c.clouddfe.google.com.internal= :37140, containerNodeURI=3Dgaming-apex-dhalperi-w-1.c.clouddfe.google.com.i= nternal:8042, containerResourceMemory1024, priority3 > 2017-04-14 18:57:11,359 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000005 > 2017-04-14 18:57:11,360 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 18:57:11,361 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000004 > 2017-04-14 18:57:11,371 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 18:57:11,372 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx805306368 for container container_1492195730173_0001_01_0= 00005 > 2017-04-14 18:57:11,372 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-1.c.clouddfe.google.com.intern= al:37140 command: $JAVA_HOME/bin/java -Xmx805306368 -Ddt.attr.APPLICATION= _PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/applic= ation_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer_1= 492195730173_0001_01_000005 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.di= r=3D com.datatorrent.stram.engine.StreamingContainer 1>/s= tdout 2>/stderr =20 > 2017-04-14 18:57:11,373 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 06, containerNode=3Dgaming-apex-dhalperi-w-16.c.clouddfe.google.com.interna= l:52928, containerNodeURI=3Dgaming-apex-dhalperi-w-16.c.clouddfe.google.com= .internal:8042, containerResourceMemory2048, priority4 > 2017-04-14 18:57:11,373 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000005 > 2017-04-14 18:57:11,384 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000006 > 2017-04-14 18:57:11,384 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 18:57:11,387 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 18:57:11,388 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000006 > 2017-04-14 18:57:11,388 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-16.c.clouddfe.google.com.inter= nal:52928 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATI= ON_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appl= ication_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer= _1492195730173_0001_01_000006 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.= dir=3D com.datatorrent.stram.engine.StreamingContainer 1>= /stdout 2>/stderr =20 > 2017-04-14 18:57:11,389 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 07, containerNode=3Dgaming-apex-dhalperi-w-12.c.clouddfe.google.com.interna= l:55762, containerNodeURI=3Dgaming-apex-dhalperi-w-12.c.clouddfe.google.com= .internal:8042, containerResourceMemory2048, priority5 > 2017-04-14 18:57:11,394 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000007 > 2017-04-14 18:57:11,394 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 18:57:11,401 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000006 > 2017-04-14 18:57:11,401 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 18:57:11,406 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000007 > 2017-04-14 18:57:11,407 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-12.c.clouddfe.google.com.inter= nal:55762 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATI= ON_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appl= ication_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer= _1492195730173_0001_01_000007 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.= dir=3D com.datatorrent.stram.engine.StreamingContainer 1>= /stdout 2>/stderr =20 > 2017-04-14 18:57:11,407 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 08, containerNode=3Dgaming-apex-dhalperi-w-3.c.clouddfe.google.com.internal= :44743, containerNodeURI=3Dgaming-apex-dhalperi-w-3.c.clouddfe.google.com.i= nternal:8042, containerResourceMemory2048, priority6 > 2017-04-14 18:57:11,412 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-1.c.clo= uddfe.google.com.internal:37140 > 2017-04-14 18:57:11,413 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000007 > 2017-04-14 18:57:11,415 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000008 > 2017-04-14 18:57:11,415 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 18:57:11,418 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-5.c.clo= uddfe.google.com.internal:33276 > 2017-04-14 18:57:11,421 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 18:57:11,422 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000008 > 2017-04-14 18:57:11,422 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-3.c.clouddfe.google.com.intern= al:44743 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATIO= N_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appli= cation_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer_= 1492195730173_0001_01_000008 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.d= ir=3D com.datatorrent.stram.engine.StreamingContainer 1>/= stdout 2>/stderr =20 > 2017-04-14 18:57:11,422 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 09, containerNode=3Dgaming-apex-dhalperi-w-2.c.clouddfe.google.com.internal= :46370, containerNodeURI=3Dgaming-apex-dhalperi-w-2.c.clouddfe.google.com.i= nternal:8042, containerResourceMemory2048, priority7 > 2017-04-14 18:57:11,426 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000008 > 2017-04-14 18:57:11,430 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000009 > 2017-04-14 18:57:11,430 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 18:57:11,433 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-11.c.cl= ouddfe.google.com.internal:50931 > 2017-04-14 18:57:11,436 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 18:57:11,436 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000009 > 2017-04-14 18:57:11,436 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-2.c.clouddfe.google.com.intern= al:46370 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATIO= N_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appli= cation_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer_= 1492195730173_0001_01_000009 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.d= ir=3D com.datatorrent.stram.engine.StreamingContainer 1>/= stdout 2>/stderr =20 > 2017-04-14 18:57:11,437 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 10, containerNode=3Dgaming-apex-dhalperi-w-15.c.clouddfe.google.com.interna= l:53054, containerNodeURI=3Dgaming-apex-dhalperi-w-15.c.clouddfe.google.com= .internal:8042, containerResourceMemory2048, priority8 > 2017-04-14 18:57:11,437 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000009 > 2017-04-14 18:57:11,449 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-3.c.clo= uddfe.google.com.internal:44743 > 2017-04-14 18:57:11,449 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000010 > 2017-04-14 18:57:11,450 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 18:57:11,454 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-12.c.cl= ouddfe.google.com.internal:55762 > 2017-04-14 18:57:11,454 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 18:57:11,455 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000010 > 2017-04-14 18:57:11,455 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-15.c.clouddfe.google.com.inter= nal:53054 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATI= ON_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appl= ication_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer= _1492195730173_0001_01_000010 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.= dir=3D com.datatorrent.stram.engine.StreamingContainer 1>= /stdout 2>/stderr =20 > 2017-04-14 18:57:11,455 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 11, containerNode=3Dgaming-apex-dhalperi-w-18.c.clouddfe.google.com.interna= l:39804, containerNodeURI=3Dgaming-apex-dhalperi-w-18.c.clouddfe.google.com= .internal:8042, containerResourceMemory2048, priority9 > 2017-04-14 18:57:11,459 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-16.c.cl= ouddfe.google.com.internal:52928 > 2017-04-14 18:57:11,463 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000011 > 2017-04-14 18:57:11,463 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 18:57:11,460 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000010 > 2017-04-14 18:57:11,473 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-2.c.clo= uddfe.google.com.internal:46370 > 2017-04-14 18:57:11,474 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-15.c.cl= ouddfe.google.com.internal:53054 > 2017-04-14 18:57:11,482 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 18:57:11,482 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000011 > 2017-04-14 18:57:11,482 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-18.c.clouddfe.google.com.inter= nal:39804 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATI= ON_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appl= ication_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer= _1492195730173_0001_01_000011 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.= dir=3D com.datatorrent.stram.engine.StreamingContainer 1>= /stdout 2>/stderr =20 > 2017-04-14 18:57:11,490 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000011 > 2017-04-14 18:57:11,500 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-18.c.cl= ouddfe.google.com.internal:39804 > 2017-04-14 18:57:13,756 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000010] Entering heartbeat = loop.. context: PTContainer[id=3D7(container_1492195730173_0001_01_000010),= state=3DALLOCATED,operators=3D[PTOperator[id=3D7,name=3DSumTeamScores/Combi= ne.perKey(SumInteger)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Ano= nymous)]]] > 2017-04-14 18:57:14,770 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000010 buffer server: gaming= -apex-dhalperi-w-15:55080 > 2017-04-14 18:57:14,941 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000008] Entering heartbeat = loop.. context: PTContainer[id=3D6(container_1492195730173_0001_01_000008),= state=3DALLOCATED,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combi= ne.perKey(SumInteger)/GroupByKey]]] > 2017-04-14 18:57:15,054 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000002] Entering heartbeat = loop.. context: PTContainer[id=3D9(container_1492195730173_0001_01_000002),= state=3DALLOCATED,operators=3D[PTOperator[id=3D9,name=3DSumTeamScores/Group= ByKey]]] > 2017-04-14 18:57:15,073 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000009] Entering heartbeat = loop.. context: PTContainer[id=3D5(container_1492195730173_0001_01_000009),= state=3DALLOCATED,operators=3D[PTOperator[id=3D5,name=3DSumTeamScores/ParDo= (KeyScoreByTeam)/ParMultiDo(KeyScoreByTeam)]]] > 2017-04-14 18:57:15,106 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000003] Entering heartbeat = loop.. context: PTContainer[id=3D1(container_1492195730173_0001_01_000003),= state=3DALLOCATED,operators=3D[PTOperator[id=3D1,name=3DTextIO.Read/Read]]] > 2017-04-14 18:57:15,125 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000011] Entering heartbeat = loop.. context: PTContainer[id=3D3(container_1492195730173_0001_01_000011),= state=3DALLOCATED,operators=3D[PTOperator[id=3D3,name=3DSetTimestamps/ParMu= ltiDo(SetTimestamps)]]] > 2017-04-14 18:57:15,232 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000006] Entering heartbeat = loop.. context: PTContainer[id=3D2(container_1492195730173_0001_01_000006),= state=3DALLOCATED,operators=3D[PTOperator[id=3D2,name=3DParseGameEvent/ParM= ultiDo(ParseEvent)]]] > 2017-04-14 18:57:15,333 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000007] Entering heartbeat = loop.. context: PTContainer[id=3D8(container_1492195730173_0001_01_000007),= state=3DALLOCATED,operators=3D[PTOperator[id=3D8,name=3DSumTeamScores/ParDo= (KeyByWindow)/ParMultiDo(KeyByWindow)]]] > 2017-04-14 18:57:15,346 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000004] Entering heartbeat = loop.. context: PTContainer[id=3D4(container_1492195730173_0001_01_000004),= state=3DALLOCATED,operators=3D[PTOperator[id=3D4,name=3DFixedWindows/Window= .Assign]]] > 2017-04-14 18:57:15,472 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000005] Entering heartbeat = loop.. context: PTContainer[id=3D10(container_1492195730173_0001_01_000005)= ,state=3DALLOCATED,operators=3D[PTOperator[id=3D10,name=3DSumTeamScores/Par= Do(WriteWindowedFiles)/ParMultiDo(WriteWindowedFiles)]]] > 2017-04-14 18:57:15,954 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000008 buffer server: gaming= -apex-dhalperi-w-3:60148 > 2017-04-14 18:57:16,066 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000002 buffer server: gaming= -apex-dhalperi-w-9:52966 > 2017-04-14 18:57:16,085 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000009 buffer server: gaming= -apex-dhalperi-w-2:41280 > 2017-04-14 18:57:16,116 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000003 buffer server: gaming= -apex-dhalperi-w-11:41089 > 2017-04-14 18:57:16,136 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000011 buffer server: gaming= -apex-dhalperi-w-18:50437 > 2017-04-14 18:57:16,244 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000006 buffer server: gaming= -apex-dhalperi-w-16:55760 > 2017-04-14 18:57:16,345 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000007 buffer server: gaming= -apex-dhalperi-w-12:42777 > 2017-04-14 18:57:16,359 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000004 buffer server: gaming= -apex-dhalperi-w-5:58095 > 2017-04-14 18:57:16,485 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000005 buffer server: gaming= -apex-dhalperi-w-1:55004 > 2017-04-14 19:00:14,948 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D8,name=3DSumTeamScores/ParDo(KeyByWin= dow)/ParMultiDo(KeyByWindow)] blocked committed window 58f11b740000003b, re= covery window 58f11b740000003b, current time 1492196414947, last window id = change time 1492196354898, window processing timeout millis 60000 > 2017-04-14 19:00:14,948 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D9,name=3DSumTeamScores/GroupByKey] bl= ocked committed window 58f11b740000003b, recovery window 58f11b740000003b, = current time 1492196414947, last window id change time 1492196354896, windo= w processing timeout millis 60000 > 2017-04-14 19:00:15,950 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D6,name=3DSumTeamScores/Combine.perKey= (SumInteger)/GroupByKey] blocked committed window 58f11b740000003b, recover= y window 58f11b740000003b, current time 1492196415950, last window id chang= e time 1492196355249, window processing timeout millis 60000 > 2017-04-14 19:00:15,950 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D7,name=3DSumTeamScores/Combine.perKey= (SumInteger)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anonymous)] = blocked committed window 58f11b740000003b, recovery window 58f11b740000003b= , current time 1492196415950, last window id change time 1492196355318, win= dow processing timeout millis 60000 > 2017-04-14 19:00:15,950 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D8,name=3DSumTeamScores/ParDo(KeyByWin= dow)/ParMultiDo(KeyByWindow)] blocked committed window 58f11b740000003b, re= covery window 58f11b740000003b, current time 1492196415950, last window id = change time 1492196354898, window processing timeout millis 60000 > 2017-04-14 19:00:15,950 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D9,name=3DSumTeamScores/GroupByKey] bl= ocked committed window 58f11b740000003b, recovery window 58f11b740000003b, = current time 1492196415950, last window id change time 1492196354896, windo= w processing timeout millis 60000 > 2017-04-14 19:00:15,950 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D10,name=3DSumTeamScores/ParDo(WriteWi= ndowedFiles)/ParMultiDo(WriteWindowedFiles)] blocked committed window 58f11= b740000003b, recovery window 58f11b740000003b, current time 1492196415950, = last window id change time 1492196354966, window processing timeout millis = 60000 > 2017-04-14 19:00:15,951 INFO com.datatorrent.stram.StreamingContainerMana= ger: Blocked operator PTOperator[id=3D6,name=3DSumTeamScores/Combine.perKey= (SumInteger)/GroupByKey] container PTContainer[id=3D6(container_14921957301= 73_0001_01_000008),state=3DACTIVE] time 60701ms > 2017-04-14 19:00:16,951 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested stop container container_1492195730173_0001_01_000008 > 2017-04-14 19:00:16,951 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: STOP_CONTAINER for Containe= r container_1492195730173_0001_01_000008 > 2017-04-14 19:00:16,953 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D6,name=3DSumTeamScores/Combine.perKey= (SumInteger)/GroupByKey] blocked committed window 58f11b740000003b, recover= y window 58f11b740000003b, current time 1492196416953, last window id chang= e time 1492196355249, window processing timeout millis 60000 > 2017-04-14 19:00:16,953 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D7,name=3DSumTeamScores/Combine.perKey= (SumInteger)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anonymous)] = blocked committed window 58f11b740000003b, recovery window 58f11b740000003b= , current time 1492196416953, last window id change time 1492196355318, win= dow processing timeout millis 60000 > 2017-04-14 19:00:16,954 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D8,name=3DSumTeamScores/ParDo(KeyByWin= dow)/ParMultiDo(KeyByWindow)] blocked committed window 58f11b740000003b, re= covery window 58f11b740000003b, current time 1492196416953, last window id = change time 1492196354898, window processing timeout millis 60000 > 2017-04-14 19:00:16,954 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D9,name=3DSumTeamScores/GroupByKey] bl= ocked committed window 58f11b740000003b, recovery window 58f11b740000003b, = current time 1492196416953, last window id change time 1492196354896, windo= w processing timeout millis 60000 > 2017-04-14 19:00:16,954 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D10,name=3DSumTeamScores/ParDo(WriteWi= ndowedFiles)/ParMultiDo(WriteWindowedFiles)] blocked committed window 58f11= b740000003b, recovery window 58f11b740000003b, current time 1492196416953, = last window id change time 1492196354966, window processing timeout millis = 60000 > 2017-04-14 19:00:16,954 INFO com.datatorrent.stram.StreamingContainerMana= ger: Blocked operator PTOperator[id=3D6,name=3DSumTeamScores/Combine.perKey= (SumInteger)/GroupByKey] container PTContainer[id=3D6(container_14921957301= 73_0001_01_000008),state=3DACTIVE] time 61704ms > 2017-04-14 19:00:16,957 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-3.c.clo= uddfe.google.com.internal:44743 > 2017-04-14 19:00:17,962 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Completed containerId=3Dcontainer_1492195730173_0001_01_000008, state= =3DCOMPLETE, exitStatus=3D-105, diagnostics=3DContainer killed by the Appli= cationMaster. > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > 2017-04-14 19:00:17,962 INFO com.datatorrent.stram.StreamingContainerMana= ger: Initiating recovery for container_1492195730173_0001_01_000008@gaming-= apex-dhalperi-w-3.c.clouddfe.google.com.internal:44743 > 2017-04-14 19:00:17,965 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D6,name=3DSumTeamScores/Combine.perKey= (SumInteger)/GroupByKey] blocked committed window 58f11b740000003b, recover= y window 58f11b740000003b, current time 1492196417965, last window id chang= e time 1492196355249, window processing timeout millis 60000 > 2017-04-14 19:00:17,965 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D7,name=3DSumTeamScores/Combine.perKey= (SumInteger)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anonymous)] = blocked committed window 58f11b740000003b, recovery window 58f11b740000003b= , current time 1492196417965, last window id change time 1492196355318, win= dow processing timeout millis 60000 > 2017-04-14 19:00:17,965 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D8,name=3DSumTeamScores/ParDo(KeyByWin= dow)/ParMultiDo(KeyByWindow)] blocked committed window 58f11b740000003b, re= covery window 58f11b740000003b, current time 1492196417965, last window id = change time 1492196354898, window processing timeout millis 60000 > 2017-04-14 19:00:17,965 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D9,name=3DSumTeamScores/GroupByKey] bl= ocked committed window 58f11b740000003b, recovery window 58f11b740000003b, = current time 1492196417965, last window id change time 1492196354896, windo= w processing timeout millis 60000 > 2017-04-14 19:00:17,965 WARN com.datatorrent.stram.StreamingContainerMana= ger: Marking operator PTOperator[id=3D10,name=3DSumTeamScores/ParDo(WriteWi= ndowedFiles)/ParMultiDo(WriteWindowedFiles)] blocked committed window 58f11= b740000003b, recovery window 58f11b740000003b, current time 1492196417965, = last window id change time 1492196354966, window processing timeout millis = 60000 > 2017-04-14 19:00:17,965 INFO com.datatorrent.stram.StreamingContainerMana= ger: Affected operators [PTOperator[id=3D6,name=3DSumTeamScores/Combine.per= Key(SumInteger)/GroupByKey], PTOperator[id=3D7,name=3DSumTeamScores/Combine= .perKey(SumInteger)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anony= mous)], PTOperator[id=3D8,name=3DSumTeamScores/ParDo(KeyByWindow)/ParMultiD= o(KeyByWindow)], PTOperator[id=3D9,name=3DSumTeamScores/GroupByKey], PTOper= ator[id=3D10,name=3DSumTeamScores/ParDo(WriteWindowedFiles)/ParMultiDo(Writ= eWindowedFiles)]] > 2017-04-14 19:00:19,013 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 6,name=3DSumTeamScores/Combine.perKey(SumInteger)/GroupByKey] > 2017-04-14 19:00:19,013 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 19:00:19,014 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Asking RM for containers: [Capability[]Priority= [10]] > 2017-04-14 19:00:19,014 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[10] o= n host: [null] > 2017-04-14 19:00:20,021 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 12, containerNode=3Dgaming-apex-dhalperi-w-3.c.clouddfe.google.com.internal= :44743, containerNodeURI=3Dgaming-apex-dhalperi-w-3.c.clouddfe.google.com.i= nternal:8042, containerResourceMemory2048, priority10 > 2017-04-14 19:00:20,022 INFO com.datatorrent.stram.StreamingContainerMana= ger: Removing container agent container_1492195730173_0001_01_000008 > 2017-04-14 19:00:20,024 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000012 > 2017-04-14 19:00:20,024 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 19:00:20,034 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 19:00:20,035 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000012 > 2017-04-14 19:00:20,035 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-3.c.clouddfe.google.com.intern= al:44743 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATIO= N_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appli= cation_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer_= 1492195730173_0001_01_000012 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.d= ir=3D com.datatorrent.stram.engine.StreamingContainer 1>/= stdout 2>/stderr =20 > 2017-04-14 19:00:20,035 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000012 > 2017-04-14 19:00:20,036 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-3.c.clo= uddfe.google.com.internal:44743 > 2017-04-14 19:00:21,762 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000012] Entering heartbeat = loop.. context: PTContainer[id=3D6(container_1492195730173_0001_01_000012),= state=3DALLOCATED,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combi= ne.perKey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:22,774 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000012 buffer server: gaming= -apex-dhalperi-w-3:54029 > 2017-04-14 19:00:23,006 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: deploy request failed: [OperatorDeployInfo[id=3D6,name=3DSum= TeamScores/Combine.perKey(SumInteger)/GroupByKey,type=3DGENERIC,checkpoint= =3D{58f11b740000003b, 0, 0},inputs=3D[OperatorDeployInfo.InputDeployInfo[po= rtName=3Dinput,streamId=3Dstream7,sourceNodeId=3D5,sourcePortName=3Doutput,= locality=3D,partitionMask=3D0,partitionKeys=3D]],outputs=3D[Ope= ratorDeployInfo.OutputDeployInfo[portName=3Doutput,streamId=3Dstream4,buffe= rServer=3Dgaming-apex-dhalperi-w-3]]]] com.esotericsoftware.kryo.KryoExcept= ion: Class cannot be created (missing no-arg constructor): org.apache.beam.= runners.core.AutoValue_TimerInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > context: PTContainer[id=3D6(container_1492195730173_0001_01_000012),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:23,011 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: java.lang.IllegalStateException: Deploy request failed: [Ope= ratorDeployInfo[id=3D6,name=3DSumTeamScores/Combine.perKey(SumInteger)/Grou= pByKey,type=3DGENERIC,checkpoint=3D{58f11b740000003b, 0, 0},inputs=3D[Opera= torDeployInfo.InputDeployInfo[portName=3Dinput,streamId=3Dstream7,sourceNod= eId=3D5,sourcePortName=3Doutput,locality=3D,partitionMask=3D0,partiti= onKeys=3D]],outputs=3D[OperatorDeployInfo.OutputDeployInfo[portName= =3Doutput,streamId=3Dstream4,bufferServer=3Dgaming-apex-dhalperi-w-3]]]] > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:829) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > Caused by: com.esotericsoftware.kryo.KryoException: Class cannot be creat= ed (missing no-arg constructor): org.apache.beam.runners.core.AutoValue_Tim= erInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09... 2 more > context: PTContainer[id=3D6(container_1492195730173_0001_01_000012),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:24,048 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Completed containerId=3Dcontainer_1492195730173_0001_01_000012, state= =3DCOMPLETE, exitStatus=3D1, diagnostics=3DException from container-launch. > Container id: container_1492195730173_0001_01_000012 > Exit code: 1 > Stack trace: ExitCodeException exitCode=3D1:=20 > =09at org.apache.hadoop.util.Shell.runCommand(Shell.java:582) > =09at org.apache.hadoop.util.Shell.run(Shell.java:479) > =09at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.jav= a:773) > =09at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.= launchContainer(DefaultContainerExecutor.java:212) > =09at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher= .ContainerLaunch.call(ContainerLaunch.java:302) > =09at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher= .ContainerLaunch.call(ContainerLaunch.java:82) > =09at java.util.concurrent.FutureTask.run(FutureTask.java:266) > =09at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto= r.java:1142) > =09at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecut= or.java:617) > =09at java.lang.Thread.run(Thread.java:745) > Container exited with a non-zero exit code 1 > 2017-04-14 19:00:24,049 INFO com.datatorrent.stram.StreamingContainerMana= ger: Initiating recovery for container_1492195730173_0001_01_000012@gaming-= apex-dhalperi-w-3.c.clouddfe.google.com.internal:44743 > 2017-04-14 19:00:24,049 INFO com.datatorrent.stram.StreamingContainerMana= ger: Affected operators [PTOperator[id=3D6,name=3DSumTeamScores/Combine.per= Key(SumInteger)/GroupByKey], PTOperator[id=3D7,name=3DSumTeamScores/Combine= .perKey(SumInteger)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anony= mous)], PTOperator[id=3D8,name=3DSumTeamScores/ParDo(KeyByWindow)/ParMultiD= o(KeyByWindow)], PTOperator[id=3D9,name=3DSumTeamScores/GroupByKey], PTOper= ator[id=3D10,name=3DSumTeamScores/ParDo(WriteWindowedFiles)/ParMultiDo(Writ= eWindowedFiles)]] > 2017-04-14 19:00:25,072 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 6,name=3DSumTeamScores/Combine.perKey(SumInteger)/GroupByKey] > 2017-04-14 19:00:25,072 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 19:00:25,072 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Asking RM for containers: [Capability[]Priority= [11]] > 2017-04-14 19:00:25,072 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[11] o= n host: [null] > 2017-04-14 19:00:26,078 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 13, containerNode=3Dgaming-apex-dhalperi-w-16.c.clouddfe.google.com.interna= l:52928, containerNodeURI=3Dgaming-apex-dhalperi-w-16.c.clouddfe.google.com= .internal:8042, containerResourceMemory2048, priority11 > 2017-04-14 19:00:26,079 INFO com.datatorrent.stram.StreamingContainerMana= ger: Removing container agent container_1492195730173_0001_01_000012 > 2017-04-14 19:00:26,081 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000013 > 2017-04-14 19:00:26,081 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 19:00:26,084 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 19:00:26,084 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000013 > 2017-04-14 19:00:26,085 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-16.c.clouddfe.google.com.inter= nal:52928 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATI= ON_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appl= ication_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer= _1492195730173_0001_01_000013 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.= dir=3D com.datatorrent.stram.engine.StreamingContainer 1>= /stdout 2>/stderr =20 > 2017-04-14 19:00:26,085 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000013 > 2017-04-14 19:00:26,096 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-16.c.cl= ouddfe.google.com.internal:52928 > 2017-04-14 19:00:30,185 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000013] Entering heartbeat = loop.. context: PTContainer[id=3D6(container_1492195730173_0001_01_000013),= state=3DALLOCATED,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combi= ne.perKey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:31,201 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000013 buffer server: gaming= -apex-dhalperi-w-16:43881 > 2017-04-14 19:00:31,590 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: deploy request failed: [OperatorDeployInfo[id=3D6,name=3DSum= TeamScores/Combine.perKey(SumInteger)/GroupByKey,type=3DGENERIC,checkpoint= =3D{58f11b740000003b, 0, 0},inputs=3D[OperatorDeployInfo.InputDeployInfo[po= rtName=3Dinput,streamId=3Dstream7,sourceNodeId=3D5,sourcePortName=3Doutput,= locality=3D,partitionMask=3D0,partitionKeys=3D]],outputs=3D[Ope= ratorDeployInfo.OutputDeployInfo[portName=3Doutput,streamId=3Dstream4,buffe= rServer=3Dgaming-apex-dhalperi-w-16]]]] com.esotericsoftware.kryo.KryoExcep= tion: Class cannot be created (missing no-arg constructor): org.apache.beam= .runners.core.AutoValue_TimerInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > context: PTContainer[id=3D6(container_1492195730173_0001_01_000013),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:31,618 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: java.lang.IllegalStateException: Deploy request failed: [Ope= ratorDeployInfo[id=3D6,name=3DSumTeamScores/Combine.perKey(SumInteger)/Grou= pByKey,type=3DGENERIC,checkpoint=3D{58f11b740000003b, 0, 0},inputs=3D[Opera= torDeployInfo.InputDeployInfo[portName=3Dinput,streamId=3Dstream7,sourceNod= eId=3D5,sourcePortName=3Doutput,locality=3D,partitionMask=3D0,partiti= onKeys=3D]],outputs=3D[OperatorDeployInfo.OutputDeployInfo[portName= =3Doutput,streamId=3Dstream4,bufferServer=3Dgaming-apex-dhalperi-w-16]]]] > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:829) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > Caused by: com.esotericsoftware.kryo.KryoException: Class cannot be creat= ed (missing no-arg constructor): org.apache.beam.runners.core.AutoValue_Tim= erInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09... 2 more > context: PTContainer[id=3D6(container_1492195730173_0001_01_000013),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:34,104 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Completed containerId=3Dcontainer_1492195730173_0001_01_000013, state= =3DCOMPLETE, exitStatus=3D1, diagnostics=3DException from container-launch. > Container id: container_1492195730173_0001_01_000013 > Exit code: 1 > Stack trace: ExitCodeException exitCode=3D1:=20 > =09at org.apache.hadoop.util.Shell.runCommand(Shell.java:582) > =09at org.apache.hadoop.util.Shell.run(Shell.java:479) > =09at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.jav= a:773) > =09at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.= launchContainer(DefaultContainerExecutor.java:212) > =09at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher= .ContainerLaunch.call(ContainerLaunch.java:302) > =09at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher= .ContainerLaunch.call(ContainerLaunch.java:82) > =09at java.util.concurrent.FutureTask.run(FutureTask.java:266) > =09at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto= r.java:1142) > =09at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecut= or.java:617) > =09at java.lang.Thread.run(Thread.java:745) > Container exited with a non-zero exit code 1 > 2017-04-14 19:00:34,105 INFO com.datatorrent.stram.StreamingContainerMana= ger: Initiating recovery for container_1492195730173_0001_01_000013@gaming-= apex-dhalperi-w-16.c.clouddfe.google.com.internal:52928 > 2017-04-14 19:00:34,105 INFO com.datatorrent.stram.StreamingContainerMana= ger: Affected operators [PTOperator[id=3D6,name=3DSumTeamScores/Combine.per= Key(SumInteger)/GroupByKey], PTOperator[id=3D7,name=3DSumTeamScores/Combine= .perKey(SumInteger)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anony= mous)], PTOperator[id=3D8,name=3DSumTeamScores/ParDo(KeyByWindow)/ParMultiD= o(KeyByWindow)], PTOperator[id=3D9,name=3DSumTeamScores/GroupByKey], PTOper= ator[id=3D10,name=3DSumTeamScores/ParDo(WriteWindowedFiles)/ParMultiDo(Writ= eWindowedFiles)]] > 2017-04-14 19:00:35,127 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 6,name=3DSumTeamScores/Combine.perKey(SumInteger)/GroupByKey] > 2017-04-14 19:00:35,127 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 19:00:35,127 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Asking RM for containers: [Capability[]Priority= [12]] > 2017-04-14 19:00:35,127 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[12] o= n host: [null] > 2017-04-14 19:00:36,135 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 14, containerNode=3Dgaming-apex-dhalperi-w-15.c.clouddfe.google.com.interna= l:53054, containerNodeURI=3Dgaming-apex-dhalperi-w-15.c.clouddfe.google.com= .internal:8042, containerResourceMemory2048, priority12 > 2017-04-14 19:00:36,135 INFO com.datatorrent.stram.StreamingContainerMana= ger: Removing container agent container_1492195730173_0001_01_000013 > 2017-04-14 19:00:36,138 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000014 > 2017-04-14 19:00:36,138 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 19:00:36,144 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 19:00:36,144 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000014 > 2017-04-14 19:00:36,144 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-15.c.clouddfe.google.com.inter= nal:53054 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATI= ON_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appl= ication_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer= _1492195730173_0001_01_000014 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.= dir=3D com.datatorrent.stram.engine.StreamingContainer 1>= /stdout 2>/stderr =20 > 2017-04-14 19:00:36,144 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000014 > 2017-04-14 19:00:36,145 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-15.c.cl= ouddfe.google.com.internal:53054 > 2017-04-14 19:00:37,902 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000014] Entering heartbeat = loop.. context: PTContainer[id=3D6(container_1492195730173_0001_01_000014),= state=3DALLOCATED,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combi= ne.perKey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:38,913 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000014 buffer server: gaming= -apex-dhalperi-w-15:59584 > 2017-04-14 19:00:39,107 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: deploy request failed: [OperatorDeployInfo[id=3D6,name=3DSum= TeamScores/Combine.perKey(SumInteger)/GroupByKey,type=3DGENERIC,checkpoint= =3D{58f11b740000003b, 0, 0},inputs=3D[OperatorDeployInfo.InputDeployInfo[po= rtName=3Dinput,streamId=3Dstream7,sourceNodeId=3D5,sourcePortName=3Doutput,= locality=3D,partitionMask=3D0,partitionKeys=3D]],outputs=3D[Ope= ratorDeployInfo.OutputDeployInfo[portName=3Doutput,streamId=3Dstream4,buffe= rServer=3Dgaming-apex-dhalperi-w-15]]]] com.esotericsoftware.kryo.KryoExcep= tion: Class cannot be created (missing no-arg constructor): org.apache.beam= .runners.core.AutoValue_TimerInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > context: PTContainer[id=3D6(container_1492195730173_0001_01_000014),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:39,112 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: java.lang.IllegalStateException: Deploy request failed: [Ope= ratorDeployInfo[id=3D6,name=3DSumTeamScores/Combine.perKey(SumInteger)/Grou= pByKey,type=3DGENERIC,checkpoint=3D{58f11b740000003b, 0, 0},inputs=3D[Opera= torDeployInfo.InputDeployInfo[portName=3Dinput,streamId=3Dstream7,sourceNod= eId=3D5,sourcePortName=3Doutput,locality=3D,partitionMask=3D0,partiti= onKeys=3D]],outputs=3D[OperatorDeployInfo.OutputDeployInfo[portName= =3Doutput,streamId=3Dstream4,bufferServer=3Dgaming-apex-dhalperi-w-15]]]] > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:829) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > Caused by: com.esotericsoftware.kryo.KryoException: Class cannot be creat= ed (missing no-arg constructor): org.apache.beam.runners.core.AutoValue_Tim= erInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09... 2 more > context: PTContainer[id=3D6(container_1492195730173_0001_01_000014),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:40,160 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Completed containerId=3Dcontainer_1492195730173_0001_01_000014, state= =3DCOMPLETE, exitStatus=3D1, diagnostics=3DException from container-launch. > Container id: container_1492195730173_0001_01_000014 > Exit code: 1 > Stack trace: ExitCodeException exitCode=3D1:=20 > =09at org.apache.hadoop.util.Shell.runCommand(Shell.java:582) > =09at org.apache.hadoop.util.Shell.run(Shell.java:479) > =09at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.jav= a:773) > =09at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.= launchContainer(DefaultContainerExecutor.java:212) > =09at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher= .ContainerLaunch.call(ContainerLaunch.java:302) > =09at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher= .ContainerLaunch.call(ContainerLaunch.java:82) > =09at java.util.concurrent.FutureTask.run(FutureTask.java:266) > =09at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto= r.java:1142) > =09at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecut= or.java:617) > =09at java.lang.Thread.run(Thread.java:745) > Container exited with a non-zero exit code 1 > 2017-04-14 19:00:40,161 INFO com.datatorrent.stram.StreamingContainerMana= ger: Initiating recovery for container_1492195730173_0001_01_000014@gaming-= apex-dhalperi-w-15.c.clouddfe.google.com.internal:53054 > 2017-04-14 19:00:40,161 INFO com.datatorrent.stram.StreamingContainerMana= ger: Affected operators [PTOperator[id=3D6,name=3DSumTeamScores/Combine.per= Key(SumInteger)/GroupByKey], PTOperator[id=3D7,name=3DSumTeamScores/Combine= .perKey(SumInteger)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anony= mous)], PTOperator[id=3D8,name=3DSumTeamScores/ParDo(KeyByWindow)/ParMultiD= o(KeyByWindow)], PTOperator[id=3D9,name=3DSumTeamScores/GroupByKey], PTOper= ator[id=3D10,name=3DSumTeamScores/ParDo(WriteWindowedFiles)/ParMultiDo(Writ= eWindowedFiles)]] > 2017-04-14 19:00:41,199 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 6,name=3DSumTeamScores/Combine.perKey(SumInteger)/GroupByKey] > 2017-04-14 19:00:41,199 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 19:00:41,199 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Asking RM for containers: [Capability[]Priority= [13]] > 2017-04-14 19:00:41,199 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[13] o= n host: [null] > 2017-04-14 19:00:42,206 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-8.c.clouddfe.goo= gle.com.internal:53733 > 2017-04-14 19:00:42,206 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 15, containerNode=3Dgaming-apex-dhalperi-w-8.c.clouddfe.google.com.internal= :53733, containerNodeURI=3Dgaming-apex-dhalperi-w-8.c.clouddfe.google.com.i= nternal:8042, containerResourceMemory2048, priority13 > 2017-04-14 19:00:42,206 INFO com.datatorrent.stram.StreamingContainerMana= ger: Removing container agent container_1492195730173_0001_01_000014 > 2017-04-14 19:00:42,208 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000015 > 2017-04-14 19:00:42,208 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 19:00:42,212 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 19:00:42,212 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000015 > 2017-04-14 19:00:42,212 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-8.c.clouddfe.google.com.intern= al:53733 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATIO= N_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appli= cation_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer_= 1492195730173_0001_01_000015 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.d= ir=3D com.datatorrent.stram.engine.StreamingContainer 1>/= stdout 2>/stderr =20 > 2017-04-14 19:00:42,212 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Set NMClientAsync thread pool size to 21 as the number = of nodes to talk to is 11 > 2017-04-14 19:00:42,214 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000015 > 2017-04-14 19:00:42,219 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-8.c.clo= uddfe.google.com.internal:53733 > 2017-04-14 19:00:45,476 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000015] Entering heartbeat = loop.. context: PTContainer[id=3D6(container_1492195730173_0001_01_000015),= state=3DALLOCATED,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combi= ne.perKey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:46,487 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000015 buffer server: gaming= -apex-dhalperi-w-8:57550 > 2017-04-14 19:00:46,661 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: deploy request failed: [OperatorDeployInfo[id=3D6,name=3DSum= TeamScores/Combine.perKey(SumInteger)/GroupByKey,type=3DGENERIC,checkpoint= =3D{58f11b740000003b, 0, 0},inputs=3D[OperatorDeployInfo.InputDeployInfo[po= rtName=3Dinput,streamId=3Dstream7,sourceNodeId=3D5,sourcePortName=3Doutput,= locality=3D,partitionMask=3D0,partitionKeys=3D]],outputs=3D[Ope= ratorDeployInfo.OutputDeployInfo[portName=3Doutput,streamId=3Dstream4,buffe= rServer=3Dgaming-apex-dhalperi-w-8]]]] com.esotericsoftware.kryo.KryoExcept= ion: Class cannot be created (missing no-arg constructor): org.apache.beam.= runners.core.AutoValue_TimerInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > context: PTContainer[id=3D6(container_1492195730173_0001_01_000015),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:46,668 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: java.lang.IllegalStateException: Deploy request failed: [Ope= ratorDeployInfo[id=3D6,name=3DSumTeamScores/Combine.perKey(SumInteger)/Grou= pByKey,type=3DGENERIC,checkpoint=3D{58f11b740000003b, 0, 0},inputs=3D[Opera= torDeployInfo.InputDeployInfo[portName=3Dinput,streamId=3Dstream7,sourceNod= eId=3D5,sourcePortName=3Doutput,locality=3D,partitionMask=3D0,partiti= onKeys=3D]],outputs=3D[OperatorDeployInfo.OutputDeployInfo[portName= =3Doutput,streamId=3Dstream4,bufferServer=3Dgaming-apex-dhalperi-w-8]]]] > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:829) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > Caused by: com.esotericsoftware.kryo.KryoException: Class cannot be creat= ed (missing no-arg constructor): org.apache.beam.runners.core.AutoValue_Tim= erInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09... 2 more > context: PTContainer[id=3D6(container_1492195730173_0001_01_000015),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:47,225 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Completed containerId=3Dcontainer_1492195730173_0001_01_000015, state= =3DCOMPLETE, exitStatus=3D1, diagnostics=3DException from container-launch. > Container id: container_1492195730173_0001_01_000015 > Exit code: 1 > Stack trace: ExitCodeException exitCode=3D1:=20 > =09at org.apache.hadoop.util.Shell.runCommand(Shell.java:582) > =09at org.apache.hadoop.util.Shell.run(Shell.java:479) > =09at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.jav= a:773) > =09at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.= launchContainer(DefaultContainerExecutor.java:212) > =09at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher= .ContainerLaunch.call(ContainerLaunch.java:302) > =09at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher= .ContainerLaunch.call(ContainerLaunch.java:82) > =09at java.util.concurrent.FutureTask.run(FutureTask.java:266) > =09at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto= r.java:1142) > =09at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecut= or.java:617) > =09at java.lang.Thread.run(Thread.java:745) > Container exited with a non-zero exit code 1 > 2017-04-14 19:00:47,226 INFO com.datatorrent.stram.StreamingContainerMana= ger: Initiating recovery for container_1492195730173_0001_01_000015@gaming-= apex-dhalperi-w-8.c.clouddfe.google.com.internal:53733 > 2017-04-14 19:00:47,226 INFO com.datatorrent.stram.StreamingContainerMana= ger: Affected operators [PTOperator[id=3D6,name=3DSumTeamScores/Combine.per= Key(SumInteger)/GroupByKey], PTOperator[id=3D7,name=3DSumTeamScores/Combine= .perKey(SumInteger)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anony= mous)], PTOperator[id=3D8,name=3DSumTeamScores/ParDo(KeyByWindow)/ParMultiD= o(KeyByWindow)], PTOperator[id=3D9,name=3DSumTeamScores/GroupByKey], PTOper= ator[id=3D10,name=3DSumTeamScores/ParDo(WriteWindowedFiles)/ParMultiDo(Writ= eWindowedFiles)]] > 2017-04-14 19:00:48,263 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 6,name=3DSumTeamScores/Combine.perKey(SumInteger)/GroupByKey] > 2017-04-14 19:00:48,263 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 19:00:48,264 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Asking RM for containers: [Capability[]Priority= [14]] > 2017-04-14 19:00:48,264 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[14] o= n host: [null] > 2017-04-14 19:00:49,271 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 16, containerNode=3Dgaming-apex-dhalperi-w-16.c.clouddfe.google.com.interna= l:52928, containerNodeURI=3Dgaming-apex-dhalperi-w-16.c.clouddfe.google.com= .internal:8042, containerResourceMemory2048, priority14 > 2017-04-14 19:00:49,272 INFO com.datatorrent.stram.StreamingContainerMana= ger: Removing container agent container_1492195730173_0001_01_000015 > 2017-04-14 19:00:49,274 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000016 > 2017-04-14 19:00:49,274 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 19:00:49,277 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 19:00:49,277 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000016 > 2017-04-14 19:00:49,277 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-16.c.clouddfe.google.com.inter= nal:52928 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATI= ON_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appl= ication_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer= _1492195730173_0001_01_000016 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.= dir=3D com.datatorrent.stram.engine.StreamingContainer 1>= /stdout 2>/stderr =20 > 2017-04-14 19:00:49,278 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000016 > 2017-04-14 19:00:49,279 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-16.c.cl= ouddfe.google.com.internal:52928 > 2017-04-14 19:00:52,751 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000016] Entering heartbeat = loop.. context: PTContainer[id=3D6(container_1492195730173_0001_01_000016),= state=3DALLOCATED,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combi= ne.perKey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:53,775 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000016 buffer server: gaming= -apex-dhalperi-w-16:52403 > 2017-04-14 19:00:54,135 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: deploy request failed: [OperatorDeployInfo[id=3D6,name=3DSum= TeamScores/Combine.perKey(SumInteger)/GroupByKey,type=3DGENERIC,checkpoint= =3D{58f11b740000003b, 0, 0},inputs=3D[OperatorDeployInfo.InputDeployInfo[po= rtName=3Dinput,streamId=3Dstream7,sourceNodeId=3D5,sourcePortName=3Doutput,= locality=3D,partitionMask=3D0,partitionKeys=3D]],outputs=3D[Ope= ratorDeployInfo.OutputDeployInfo[portName=3Doutput,streamId=3Dstream4,buffe= rServer=3Dgaming-apex-dhalperi-w-16]]]] com.esotericsoftware.kryo.KryoExcep= tion: Class cannot be created (missing no-arg constructor): org.apache.beam= .runners.core.AutoValue_TimerInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > context: PTContainer[id=3D6(container_1492195730173_0001_01_000016),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:54,145 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: java.lang.IllegalStateException: Deploy request failed: [Ope= ratorDeployInfo[id=3D6,name=3DSumTeamScores/Combine.perKey(SumInteger)/Grou= pByKey,type=3DGENERIC,checkpoint=3D{58f11b740000003b, 0, 0},inputs=3D[Opera= torDeployInfo.InputDeployInfo[portName=3Dinput,streamId=3Dstream7,sourceNod= eId=3D5,sourcePortName=3Doutput,locality=3D,partitionMask=3D0,partiti= onKeys=3D]],outputs=3D[OperatorDeployInfo.OutputDeployInfo[portName= =3Doutput,streamId=3Dstream4,bufferServer=3Dgaming-apex-dhalperi-w-16]]]] > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:829) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > Caused by: com.esotericsoftware.kryo.KryoException: Class cannot be creat= ed (missing no-arg constructor): org.apache.beam.runners.core.AutoValue_Tim= erInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09... 2 more > context: PTContainer[id=3D6(container_1492195730173_0001_01_000016),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:00:55,291 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Completed containerId=3Dcontainer_1492195730173_0001_01_000016, state= =3DCOMPLETE, exitStatus=3D1, diagnostics=3DException from container-launch. > Container id: container_1492195730173_0001_01_000016 > Exit code: 1 > Stack trace: ExitCodeException exitCode=3D1:=20 > =09at org.apache.hadoop.util.Shell.runCommand(Shell.java:582) > =09at org.apache.hadoop.util.Shell.run(Shell.java:479) > =09at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.jav= a:773) > =09at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.= launchContainer(DefaultContainerExecutor.java:212) > =09at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher= .ContainerLaunch.call(ContainerLaunch.java:302) > =09at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher= .ContainerLaunch.call(ContainerLaunch.java:82) > =09at java.util.concurrent.FutureTask.run(FutureTask.java:266) > =09at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto= r.java:1142) > =09at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecut= or.java:617) > =09at java.lang.Thread.run(Thread.java:745) > Container exited with a non-zero exit code 1 > 2017-04-14 19:00:55,292 INFO com.datatorrent.stram.StreamingContainerMana= ger: Initiating recovery for container_1492195730173_0001_01_000016@gaming-= apex-dhalperi-w-16.c.clouddfe.google.com.internal:52928 > 2017-04-14 19:00:55,292 INFO com.datatorrent.stram.StreamingContainerMana= ger: Affected operators [PTOperator[id=3D6,name=3DSumTeamScores/Combine.per= Key(SumInteger)/GroupByKey], PTOperator[id=3D7,name=3DSumTeamScores/Combine= .perKey(SumInteger)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anony= mous)], PTOperator[id=3D8,name=3DSumTeamScores/ParDo(KeyByWindow)/ParMultiD= o(KeyByWindow)], PTOperator[id=3D9,name=3DSumTeamScores/GroupByKey], PTOper= ator[id=3D10,name=3DSumTeamScores/ParDo(WriteWindowedFiles)/ParMultiDo(Writ= eWindowedFiles)]] > 2017-04-14 19:00:56,328 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 6,name=3DSumTeamScores/Combine.perKey(SumInteger)/GroupByKey] > 2017-04-14 19:00:56,328 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 19:00:56,329 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Asking RM for containers: [Capability[]Priority= [15]] > 2017-04-14 19:00:56,329 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[15] o= n host: [null] > 2017-04-14 19:00:57,337 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-14.c.clouddfe.go= ogle.com.internal:56866 > 2017-04-14 19:00:57,337 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 17, containerNode=3Dgaming-apex-dhalperi-w-14.c.clouddfe.google.com.interna= l:56866, containerNodeURI=3Dgaming-apex-dhalperi-w-14.c.clouddfe.google.com= .internal:8042, containerResourceMemory2048, priority15 > 2017-04-14 19:00:57,337 INFO com.datatorrent.stram.StreamingContainerMana= ger: Removing container agent container_1492195730173_0001_01_000016 > 2017-04-14 19:00:57,339 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000017 > 2017-04-14 19:00:57,340 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 19:00:57,348 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 19:00:57,348 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000017 > 2017-04-14 19:00:57,348 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-14.c.clouddfe.google.com.inter= nal:56866 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATI= ON_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appl= ication_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer= _1492195730173_0001_01_000017 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.= dir=3D com.datatorrent.stram.engine.StreamingContainer 1>= /stdout 2>/stderr =20 > 2017-04-14 19:00:57,349 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000017 > 2017-04-14 19:00:57,354 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-14.c.cl= ouddfe.google.com.internal:56866 > 2017-04-14 19:01:01,038 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000017] Entering heartbeat = loop.. context: PTContainer[id=3D6(container_1492195730173_0001_01_000017),= state=3DALLOCATED,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combi= ne.perKey(SumInteger)/GroupByKey]]] > 2017-04-14 19:01:02,049 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000017 buffer server: gaming= -apex-dhalperi-w-14:50842 > 2017-04-14 19:01:02,198 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: deploy request failed: [OperatorDeployInfo[id=3D6,name=3DSum= TeamScores/Combine.perKey(SumInteger)/GroupByKey,type=3DGENERIC,checkpoint= =3D{58f11b740000003b, 0, 0},inputs=3D[OperatorDeployInfo.InputDeployInfo[po= rtName=3Dinput,streamId=3Dstream7,sourceNodeId=3D5,sourcePortName=3Doutput,= locality=3D,partitionMask=3D0,partitionKeys=3D]],outputs=3D[Ope= ratorDeployInfo.OutputDeployInfo[portName=3Doutput,streamId=3Dstream4,buffe= rServer=3Dgaming-apex-dhalperi-w-14]]]] com.esotericsoftware.kryo.KryoExcep= tion: Class cannot be created (missing no-arg constructor): org.apache.beam= .runners.core.AutoValue_TimerInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > context: PTContainer[id=3D6(container_1492195730173_0001_01_000017),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:01:02,204 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: java.lang.IllegalStateException: Deploy request failed: [Ope= ratorDeployInfo[id=3D6,name=3DSumTeamScores/Combine.perKey(SumInteger)/Grou= pByKey,type=3DGENERIC,checkpoint=3D{58f11b740000003b, 0, 0},inputs=3D[Opera= torDeployInfo.InputDeployInfo[portName=3Dinput,streamId=3Dstream7,sourceNod= eId=3D5,sourcePortName=3Doutput,locality=3D,partitionMask=3D0,partiti= onKeys=3D]],outputs=3D[OperatorDeployInfo.OutputDeployInfo[portName= =3Doutput,streamId=3Dstream4,bufferServer=3Dgaming-apex-dhalperi-w-14]]]] > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:829) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > Caused by: com.esotericsoftware.kryo.KryoException: Class cannot be creat= ed (missing no-arg constructor): org.apache.beam.runners.core.AutoValue_Tim= erInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09... 2 more > context: PTContainer[id=3D6(container_1492195730173_0001_01_000017),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:01:03,363 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Completed containerId=3Dcontainer_1492195730173_0001_01_000017, state= =3DCOMPLETE, exitStatus=3D1, diagnostics=3DException from container-launch. > Container id: container_1492195730173_0001_01_000017 > Exit code: 1 > Stack trace: ExitCodeException exitCode=3D1:=20 > =09at org.apache.hadoop.util.Shell.runCommand(Shell.java:582) > =09at org.apache.hadoop.util.Shell.run(Shell.java:479) > =09at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.jav= a:773) > =09at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.= launchContainer(DefaultContainerExecutor.java:212) > =09at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher= .ContainerLaunch.call(ContainerLaunch.java:302) > =09at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher= .ContainerLaunch.call(ContainerLaunch.java:82) > =09at java.util.concurrent.FutureTask.run(FutureTask.java:266) > =09at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto= r.java:1142) > =09at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecut= or.java:617) > =09at java.lang.Thread.run(Thread.java:745) > Container exited with a non-zero exit code 1 > 2017-04-14 19:01:03,363 INFO com.datatorrent.stram.StreamingContainerMana= ger: Initiating recovery for container_1492195730173_0001_01_000017@gaming-= apex-dhalperi-w-14.c.clouddfe.google.com.internal:56866 > 2017-04-14 19:01:03,364 INFO com.datatorrent.stram.StreamingContainerMana= ger: Affected operators [PTOperator[id=3D6,name=3DSumTeamScores/Combine.per= Key(SumInteger)/GroupByKey], PTOperator[id=3D7,name=3DSumTeamScores/Combine= .perKey(SumInteger)/Combine.GroupedValues/ParDo(Anonymous)/ParMultiDo(Anony= mous)], PTOperator[id=3D8,name=3DSumTeamScores/ParDo(KeyByWindow)/ParMultiD= o(KeyByWindow)], PTOperator[id=3D9,name=3DSumTeamScores/GroupByKey], PTOper= ator[id=3D10,name=3DSumTeamScores/ParDo(WriteWindowedFiles)/ParMultiDo(Writ= eWindowedFiles)]] > 2017-04-14 19:01:04,387 INFO com.datatorrent.stram.ResourceRequestHandler= : Strict anti-affinity =3D [] for container with operators PTOperator[id=3D= 6,name=3DSumTeamScores/Combine.perKey(SumInteger)/GroupByKey] > 2017-04-14 19:01:04,387 INFO com.datatorrent.stram.ResourceRequestHandler= : Found host null > 2017-04-14 19:01:04,388 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Asking RM for containers: [Capability[]Priority= [16]] > 2017-04-14 19:01:04,388 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Requested container: Capability[]Priority[16] o= n host: [null] > 2017-04-14 19:01:05,396 INFO org.apache.hadoop.yarn.client.api.impl.AMRMC= lientImpl: Received new token for : gaming-apex-dhalperi-w-6.c.clouddfe.goo= gle.com.internal:43371 > 2017-04-14 19:01:05,396 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Got new container., containerId=3Dcontainer_1492195730173_0001_01_0000= 18, containerNode=3Dgaming-apex-dhalperi-w-6.c.clouddfe.google.com.internal= :43371, containerNodeURI=3Dgaming-apex-dhalperi-w-6.c.clouddfe.google.com.i= nternal:8042, containerResourceMemory2048, priority16 > 2017-04-14 19:01:05,397 INFO com.datatorrent.stram.StreamingContainerMana= ger: Removing container agent container_1492195730173_0001_01_000017 > 2017-04-14 19:01:05,399 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Setting up container launch context for containerid=3Dcontainer_14921957= 30173_0001_01_000018 > 2017-04-14 19:01:05,399 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: CLASSPATH: ./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOM= E/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:$HADOOP_MAPRED_HOME/*:$= HADOOP_MAPRED_HOME/lib/*:$HADOOP_YARN_HOME/*:$HADOOP_YARN_HOME/lib/*':. > 2017-04-14 19:01:05,401 INFO com.datatorrent.common.util.BasicContainerOp= tConfigurator: property map for operator {-Xmx=3D768m, Generic=3Dnull} > 2017-04-14 19:01:05,401 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Jvm opts -Xmx1342177280 for container container_1492195730173_0001_01_= 000018 > 2017-04-14 19:01:05,401 INFO com.datatorrent.stram.LaunchContainerRunnabl= e: Launching on node: gaming-apex-dhalperi-w-6.c.clouddfe.google.com.intern= al:43371 command: $JAVA_HOME/bin/java -Xmx1342177280 -Ddt.attr.APPLICATIO= N_PATH=3Dhdfs://gaming-apex-dhalperi-m/user/dhalperi/datatorrent/apps/appli= cation_1492195730173_0001 -Djava.io.tmpdir=3D$PWD/tmp -Ddt.cid=3Dcontainer_= 1492195730173_0001_01_000018 -Dhadoop.root.logger=3DINFO,RFA -Dhadoop.log.d= ir=3D com.datatorrent.stram.engine.StreamingContainer 1>/= stdout 2>/stderr =20 > 2017-04-14 19:01:05,402 INFO org.apache.hadoop.yarn.client.api.async.impl= .NMClientAsyncImpl: Processing Event EventType: START_CONTAINER for Contain= er container_1492195730173_0001_01_000018 > 2017-04-14 19:01:05,407 INFO org.apache.hadoop.yarn.client.api.impl.Conta= inerManagementProtocolProxy: Opening proxy : gaming-apex-dhalperi-w-6.c.clo= uddfe.google.com.internal:43371 > 2017-04-14 19:01:09,197 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: [container_1492195730173_0001_01_000018] Entering heartbeat = loop.. context: PTContainer[id=3D6(container_1492195730173_0001_01_000018),= state=3DALLOCATED,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combi= ne.perKey(SumInteger)/GroupByKey]]] > 2017-04-14 19:01:10,209 INFO com.datatorrent.stram.StreamingContainerMana= ger: Container container_1492195730173_0001_01_000018 buffer server: gaming= -apex-dhalperi-w-6:49764 > 2017-04-14 19:01:10,377 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: deploy request failed: [OperatorDeployInfo[id=3D6,name=3DSum= TeamScores/Combine.perKey(SumInteger)/GroupByKey,type=3DGENERIC,checkpoint= =3D{58f11b740000003b, 0, 0},inputs=3D[OperatorDeployInfo.InputDeployInfo[po= rtName=3Dinput,streamId=3Dstream7,sourceNodeId=3D5,sourcePortName=3Doutput,= locality=3D,partitionMask=3D0,partitionKeys=3D]],outputs=3D[Ope= ratorDeployInfo.OutputDeployInfo[portName=3Doutput,streamId=3Dstream4,buffe= rServer=3Dgaming-apex-dhalperi-w-6]]]] com.esotericsoftware.kryo.KryoExcept= ion: Class cannot be created (missing no-arg constructor): org.apache.beam.= runners.core.AutoValue_TimerInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > context: PTContainer[id=3D6(container_1492195730173_0001_01_000018),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:01:10,385 INFO com.datatorrent.stram.StreamingContainerPare= nt: child msg: java.lang.IllegalStateException: Deploy request failed: [Ope= ratorDeployInfo[id=3D6,name=3DSumTeamScores/Combine.perKey(SumInteger)/Grou= pByKey,type=3DGENERIC,checkpoint=3D{58f11b740000003b, 0, 0},inputs=3D[Opera= torDeployInfo.InputDeployInfo[portName=3Dinput,streamId=3Dstream7,sourceNod= eId=3D5,sourcePortName=3Doutput,locality=3D,partitionMask=3D0,partiti= onKeys=3D]],outputs=3D[OperatorDeployInfo.OutputDeployInfo[portName= =3Doutput,streamId=3Dstream4,bufferServer=3Dgaming-apex-dhalperi-w-6]]]] > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:829) > =09at com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(Strea= mingContainer.java:705) > =09at com.datatorrent.stram.engine.StreamingContainer.main(StreamingConta= iner.java:310) > Caused by: com.esotericsoftware.kryo.KryoException: Class cannot be creat= ed (missing no-arg constructor): org.apache.beam.runners.core.AutoValue_Tim= erInternals_TimerData > Serialization trace: > activeTimers (org.apache.beam.runners.apex.translation.operators.ApexGrou= pByKeyOperator) > =09at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInsta= ntiatorOf(Kryo.java:1228) > =09at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049) > =09at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldS= erializer.java:547) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:523) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:116) > =09at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(Col= lectionSerializer.java:22) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:143) > =09at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSeriali= zer.java:21) > =09at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679) > =09at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.= java:106) > =09at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSer= ializer.java:528) > =09at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761) > =09at com.datatorrent.common.util.FSStorageAgent.retrieve(FSStorageAgent.= java:192) > =09at com.datatorrent.common.util.FSStorageAgent.load(FSStorageAgent.java= :137) > =09at com.datatorrent.stram.engine.StreamingContainer.deployNodes(Streami= ngContainer.java:914) > =09at com.datatorrent.stram.engine.StreamingContainer.deploy(StreamingCon= tainer.java:862) > =09at com.datatorrent.stram.engine.StreamingContainer.processHeartbeatRes= ponse(StreamingContainer.java:820) > =09... 2 more > context: PTContainer[id=3D6(container_1492195730173_0001_01_000018),stat= e=3DACTIVE,operators=3D[PTOperator[id=3D6,name=3DSumTeamScores/Combine.perK= ey(SumInteger)/GroupByKey]]] > 2017-04-14 19:01:11,416 INFO com.datatorrent.stram.StreamingAppMasterServ= ice: Completed containerId=3Dcontainer_1492195730173_0001_01_000018, state= =3DCOMPLETE, exitStatus=3D1, diagnostics=3DException from container-launch. > Container id: container_1492195730173_0001_01_000018 > Exit code: 1 > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)