Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id A7F91200B88 for ; Thu, 8 Sep 2016 01:58:31 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id A673C160ACF; Wed, 7 Sep 2016 23:58:31 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 334F2160AC1 for ; Thu, 8 Sep 2016 01:58:30 +0200 (CEST) Received: (qmail 66823 invoked by uid 500); 7 Sep 2016 23:58:29 -0000 Mailing-List: contact dev-help@tephra.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tephra.incubator.apache.org Delivered-To: mailing list dev@tephra.incubator.apache.org Received: (qmail 66755 invoked by uid 99); 7 Sep 2016 23:58:29 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Sep 2016 23:58:29 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id EAA861858A9 for ; Wed, 7 Sep 2016 23:58:28 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -5.446 X-Spam-Level: X-Spam-Status: No, score=-5.446 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.426] autolearn=disabled Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 36YddX803Zz6 for ; Wed, 7 Sep 2016 23:58:24 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with SMTP id C43435FE22 for ; Wed, 7 Sep 2016 23:58:22 +0000 (UTC) Received: (qmail 61189 invoked by uid 99); 7 Sep 2016 23:58:22 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Sep 2016 23:58:22 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id DD6F42C1B7B for ; Wed, 7 Sep 2016 23:58:21 +0000 (UTC) Date: Wed, 7 Sep 2016 23:58:21 +0000 (UTC) From: "Poorna Chandra (JIRA)" To: dev@tephra.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (TEPHRA-179) Tephra transaction manager breaks on zookeeper restart MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 07 Sep 2016 23:58:31 -0000 [ https://issues.apache.org/jira/browse/TEPHRA-179?page=3Dcom.atlassia= n.jira.plugin.system.issuetabpanels:all-tabpanel ] Poorna Chandra resolved TEPHRA-179. ----------------------------------- Resolution: Fixed > Tephra transaction manager breaks on zookeeper restart > ------------------------------------------------------ > > Key: TEPHRA-179 > URL: https://issues.apache.org/jira/browse/TEPHRA-179 > Project: Tephra > Issue Type: Bug > Components: manager > Affects Versions: 0.8.0-incubating > Environment: OpenJDK 8 (JDK) on Alpine Linux 3.4 in Docker > Reporter: Francis Chuang > Assignee: Ali Anwar > Fix For: 0.9.0-incubating > > > I am running HBase 1.2.2 with Phoenix 4.8.0 with the tephra transaction s= erver in 1 docker container. In another docker container, I have Zookeeper = 3.4.8 manage by Netflix Exhibitor. > When everything first starts, I am able to create transactional table and= run transactional queries. > However, once Exhibitor restarts zookeeper and tephra reconnects to zooke= eper, it no longer works correctly.=20 > Running transactional queries result in this error: > {code} > Error: Error -1 (00000) : Error while executing SQL "CREATE TABLE my_tabl= e321 (k BIGINT PRIMARY KEY, v VARCHAR) TRANSACTIONAL=3Dtrue": Remote driver= error: RuntimeException: java.lang.Exception: Thrift error for org.apache.= tephra.distributed.TransactionServiceClient$2@2361d7ab: Internal error proc= essing startShort -> Exception: Thrift error for org.apache.tephra.distribu= ted.TransactionServiceClient$2@2361d7ab: Internal error processing startSho= rt -> TApplicationException: Internal error processing startShort > SQLState: 00000 > ErrorCode: -1 > {code} > This is the full log: > {code} > Fri Sep 2 00:26:50 UTC 2016 Starting tephra service on m9edd51-hmaster1.= m9edd51 > -f: file size (blocks) unlimited > -t: cpu time (seconds) unlimited > -d: data seg size (kb) unlimited > -s: stack size (kb) 8192 > -c: core file size (blocks) unlimited > -m: resident set size (kb) unlimited > -l: locked memory (kb) 64 > -p: processes unlimited > -n: file descriptors 65536 > -v: address space (kb) unlimited > -w: locks unlimited > -e: scheduling priority 0 > -r: real-time priority 0 > Command: /usr/lib/jvm/java-1.8-openjdk/bin/java -XX:+UseConcMarkSweepGC = -cp /opt/hbase/bin/../lib/*:/opt/hbase/bin/../conf/:/opt/hbase/phoenix-clie= nt/target/*:/opt/hbase/conf:/usr/lib/jvm/java-1.8-openjdk/lib/tools.jar:/op= t/hbase:/opt/hbase/lib/activation-1.1.jar:/opt/hbase/lib/antisamy-1.5.3.jar= :/opt/hbase/lib/aopalliance-1.0.jar:/opt/hbase/lib/apacheds-i18n-2.0.0-M15.= jar:/opt/hbase/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/hbase/lib/api= -asn1-api-1.0.0-M20.jar:/opt/hbase/lib/api-util-1.0.0-M20.jar:/opt/hbase/li= b/asm-3.1.jar:/opt/hbase/lib/avro-1.7.4.jar:/opt/hbase/lib/batik-css-1.8.ja= r:/opt/hbase/lib/batik-ext-1.8.jar:/opt/hbase/lib/batik-util-1.8.jar:/opt/h= base/lib/bsh-core-2.0b4.jar:/opt/hbase/lib/commons-beanutils-1.7.0.jar:/opt= /hbase/lib/commons-beanutils-core-1.8.3.jar:/opt/hbase/lib/commons-cli-1.2.= jar:/opt/hbase/lib/commons-codec-1.9.jar:/opt/hbase/lib/commons-collections= -3.2.2.jar:/opt/hbase/lib/commons-compress-1.4.1.jar:/opt/hbase/lib/commons= -configuration-1.6.jar:/opt/hbase/lib/commons-daemon-1.0.13.jar:/opt/hbase/= lib/commons-digester-1.8.jar:/opt/hbase/lib/commons-el-1.0.jar:/opt/hbase/l= ib/commons-fileupload-1.3.1.jar:/opt/hbase/lib/commons-httpclient-3.1.jar:/= opt/hbase/lib/commons-io-2.4.jar:/opt/hbase/lib/commons-lang-2.6.jar:/opt/h= base/lib/commons-logging-1.2.jar:/opt/hbase/lib/commons-math-2.2.jar:/opt/h= base/lib/commons-math3-3.1.1.jar:/opt/hbase/lib/commons-net-3.1.jar:/opt/hb= ase/lib/disruptor-3.3.0.jar:/opt/hbase/lib/esapi-2.1.0.1.jar:/opt/hbase/lib= /findbugs-annotations-1.3.9-1.jar:/opt/hbase/lib/guava-12.0.1.jar:/opt/hbas= e/lib/guice-3.0.jar:/opt/hbase/lib/guice-servlet-3.0.jar:/opt/hbase/lib/had= oop-annotations-2.5.1.jar:/opt/hbase/lib/hadoop-auth-2.5.1.jar:/opt/hbase/l= ib/hadoop-client-2.5.1.jar:/opt/hbase/lib/hadoop-common-2.5.1.jar:/opt/hbas= e/lib/hadoop-hdfs-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-app-2.5.= 1.jar:/opt/hbase/lib/hadoop-mapreduce-client-common-2.5.1.jar:/opt/hbase/li= b/hadoop-mapreduce-client-core-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-cl= ient-jobclient-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-shuffle-2.5= .1.jar:/opt/hbase/lib/hadoop-yarn-api-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-= client-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-common-2.5.1.jar:/opt/hbase/lib= /hadoop-yarn-server-common-2.5.1.jar:/opt/hbase/lib/hbase-annotations-1.2.2= -tests.jar:/opt/hbase/lib/hbase-annotations-1.2.2.jar:/opt/hbase/lib/hbase-= client-1.2.2.jar:/opt/hbase/lib/hbase-common-1.2.2-tests.jar:/opt/hbase/lib= /hbase-common-1.2.2.jar:/opt/hbase/lib/hbase-examples-1.2.2.jar:/opt/hbase/= lib/hbase-external-blockcache-1.2.2.jar:/opt/hbase/lib/hbase-hadoop-compat-= 1.2.2.jar:/opt/hbase/lib/hbase-hadoop2-compat-1.2.2.jar:/opt/hbase/lib/hbas= e-it-1.2.2-tests.jar:/opt/hbase/lib/hbase-it-1.2.2.jar:/opt/hbase/lib/hbase= -prefix-tree-1.2.2.jar:/opt/hbase/lib/hbase-procedure-1.2.2.jar:/opt/hbase/= lib/hbase-protocol-1.2.2.jar:/opt/hbase/lib/hbase-resource-bundle-1.2.2.jar= :/opt/hbase/lib/hbase-rest-1.2.2.jar:/opt/hbase/lib/hbase-server-1.2.2-test= s.jar:/opt/hbase/lib/hbase-server-1.2.2.jar:/opt/hbase/lib/hbase-shell-1.2.= 2.jar:/opt/hbase/lib/hbase-thrift-1.2.2.jar:/opt/hbase/lib/htrace-core-3.1.= 0-incubating.jar:/opt/hbase/lib/httpclient-4.2.5.jar:/opt/hbase/lib/httpcor= e-4.4.1.jar:/opt/hbase/lib/jackson-core-asl-1.9.13.jar:/opt/hbase/lib/jacks= on-jaxrs-1.9.13.jar:/opt/hbase/lib/jackson-mapper-asl-1.9.13.jar:/opt/hbase= /lib/jackson-xc-1.9.13.jar:/opt/hbase/lib/jamon-runtime-2.4.1.jar:/opt/hbas= e/lib/jasper-compiler-5.5.23.jar:/opt/hbase/lib/jasper-runtime-5.5.23.jar:/= opt/hbase/lib/java-xmlbuilder-0.4.jar:/opt/hbase/lib/javax.inject-1.jar:/op= t/hbase/lib/jaxb-api-2.2.2.jar:/opt/hbase/lib/jaxb-impl-2.2.3-1.jar:/opt/hb= ase/lib/jcodings-1.0.8.jar:/opt/hbase/lib/jersey-client-1.9.jar:/opt/hbase/= lib/jersey-core-1.9.jar:/opt/hbase/lib/jersey-guice-1.9.jar:/opt/hbase/lib/= jersey-json-1.9.jar:/opt/hbase/lib/jersey-server-1.9.jar:/opt/hbase/lib/jet= s3t-0.9.0.jar:/opt/hbase/lib/jettison-1.3.3.jar:/opt/hbase/lib/jetty-6.1.26= .jar:/opt/hbase/lib/jetty-sslengine-6.1.26.jar:/opt/hbase/lib/jetty-util-6.= 1.26.jar:/opt/hbase/lib/joni-2.1.2.jar:/opt/hbase/lib/jruby-complete-1.6.8.= jar:/opt/hbase/lib/jsch-0.1.42.jar:/opt/hbase/lib/jsp-2.1-6.1.14.jar:/opt/h= base/lib/jsp-api-2.1-6.1.14.jar:/opt/hbase/lib/jsr305-1.3.9.jar:/opt/hbase/= lib/junit-4.12.jar:/opt/hbase/lib/leveldbjni-all-1.8.jar:/opt/hbase/lib/lib= thrift-0.9.3.jar:/opt/hbase/lib/log4j-1.2.17.jar:/opt/hbase/lib/metrics-cor= e-2.2.0.jar:/opt/hbase/lib/nekohtml-1.9.16.jar:/opt/hbase/lib/netty-all-4.0= .23.Final.jar:/opt/hbase/lib/paranamer-2.3.jar:/opt/hbase/lib/phoenix-4.8.0= -HBase-1.2-client.jar:/opt/hbase/lib/phoenix-4.8.0-HBase-1.2-server.jar:/op= t/hbase/lib/protobuf-java-2.5.0.jar:/opt/hbase/lib/servlet-api-2.5-6.1.14.j= ar:/opt/hbase/lib/servlet-api-2.5.jar:/opt/hbase/lib/slf4j-api-1.7.7.jar:/o= pt/hbase/lib/slf4j-log4j12-1.7.5.jar:/opt/hbase/lib/snappy-java-1.0.4.1.jar= :/opt/hbase/lib/spymemcached-2.11.6.jar:/opt/hbase/lib/xalan-2.7.0.jar:/opt= /hbase/lib/xml-apis-1.3.03.jar:/opt/hbase/lib/xml-apis-ext-1.3.04.jar:/opt/= hbase/lib/xmlenc-0.52.jar:/opt/hbase/lib/xom-1.2.5.jar:/opt/hbase/lib/xz-1.= 0.jar:/opt/hbase/lib/zookeeper-3.4.6.jar:: org.apache.tephra.TransactionSer= viceMain > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in [jar:file:/opt/hbase/lib/phoenix-4.8.0-HBase-1.2-= client.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in [jar:file:/opt/hbase/lib/slf4j-log4j12-1.7.5.jar!= /org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an expla= nation. > 2016-09-02 00:26:51,571 INFO [main] zookeeper.ZooKeeper: Client environm= ent:zookeeper.version=3D3.4.6-1569965, built on 02/20/2014 09:09 GMT > 2016-09-02 00:26:51,571 INFO [main] zookeeper.ZooKeeper: Client environm= ent:host.name=3Dm9edd51-hmaster1.m9edd51 > 2016-09-02 00:26:51,571 INFO [main] zookeeper.ZooKeeper: Client environm= ent:java.version=3D1.8.0_92-internal > 2016-09-02 00:26:51,571 INFO [main] zookeeper.ZooKeeper: Client environm= ent:java.vendor=3DOracle Corporation > 2016-09-02 00:26:51,571 INFO [main] zookeeper.ZooKeeper: Client environm= ent:java.home=3D/usr/lib/jvm/java-1.8-openjdk/jre > 2016-09-02 00:26:51,571 INFO [main] zookeeper.ZooKeeper: Client environm= ent:java.class.path=3D/opt/hbase/bin/../lib/phoenix-4.8.0-HBase-1.2-server.= jar:/opt/hbase/bin/../lib/phoenix-4.8.0-HBase-1.2-client.jar:/opt/hbase/bin= /../lib/hadoop-mapreduce-client-shuffle-2.5.1.jar:/opt/hbase/bin/../lib/apa= cheds-kerberos-codec-2.0.0-M15.jar:/opt/hbase/bin/../lib/httpcore-4.4.1.jar= :/opt/hbase/bin/../lib/libthrift-0.9.3.jar:/opt/hbase/bin/../lib/hbase-hado= op2-compat-1.2.2.jar:/opt/hbase/bin/../lib/hadoop-hdfs-2.5.1.jar:/opt/hbase= /bin/../lib/hbase-server-1.2.2-tests.jar:/opt/hbase/bin/../lib/commons-coll= ections-3.2.2.jar:/opt/hbase/bin/../lib/commons-compress-1.4.1.jar:/opt/hba= se/bin/../lib/jackson-jaxrs-1.9.13.jar:/opt/hbase/bin/../lib/commons-beanut= ils-1.7.0.jar:/opt/hbase/bin/../lib/jetty-util-6.1.26.jar:/opt/hbase/bin/..= /lib/hbase-annotations-1.2.2.jar:/opt/hbase/bin/../lib/jetty-6.1.26.jar:/op= t/hbase/bin/../lib/jackson-core-asl-1.9.13.jar:/opt/hbase/bin/../lib/disrup= tor-3.3.0.jar:/opt/hbase/bin/../lib/commons-cli-1.2.jar:/opt/hbase/bin/../l= ib/hbase-thrift-1.2.2.jar:/opt/hbase/bin/../lib/xalan-2.7.0.jar:/opt/hbase/= bin/../lib/hbase-shell-1.2.2.jar:/opt/hbase/bin/../lib/jackson-xc-1.9.13.ja= r:/opt/hbase/bin/../lib/activation-1.1.jar:/opt/hbase/bin/../lib/hadoop-yar= n-server-common-2.5.1.jar:/opt/hbase/bin/../lib/jersey-server-1.9.jar:/opt/= hbase/bin/../lib/commons-net-3.1.jar:/opt/hbase/bin/../lib/xmlenc-0.52.jar:= /opt/hbase/bin/../lib/batik-ext-1.8.jar:/opt/hbase/bin/../lib/jersey-guice-= 1.9.jar:/opt/hbase/bin/../lib/spymemcached-2.11.6.jar:/opt/hbase/bin/../lib= /bsh-core-2.0b4.jar:/opt/hbase/bin/../lib/hbase-server-1.2.2.jar:/opt/hbase= /bin/../lib/hbase-rest-1.2.2.jar:/opt/hbase/bin/../lib/guava-12.0.1.jar:/op= t/hbase/bin/../lib/jetty-sslengine-6.1.26.jar:/opt/hbase/bin/../lib/xz-1.0.= jar:/opt/hbase/bin/../lib/jsch-0.1.42.jar:/opt/hbase/bin/../lib/jackson-map= per-asl-1.9.13.jar:/opt/hbase/bin/../lib/hbase-examples-1.2.2.jar:/opt/hbas= e/bin/../lib/hbase-annotations-1.2.2-tests.jar:/opt/hbase/bin/../lib/hbase-= resource-bundle-1.2.2.jar:/opt/hbase/bin/../lib/snappy-java-1.0.4.1.jar:/op= t/hbase/bin/../lib/commons-daemon-1.0.13.jar:/opt/hbase/bin/../lib/commons-= el-1.0.jar:/opt/hbase/bin/../lib/jsp-api-2.1-6.1.14.jar:/opt/hbase/bin/../l= ib/hadoop-client-2.5.1.jar:/opt/hbase/bin/../lib/hadoop-mapreduce-client-co= re-2.5.1.jar:/opt/hbase/bin/../lib/servlet-api-2.5.jar:/opt/hbase/bin/../li= b/httpclient-4.2.5.jar:/opt/hbase/bin/../lib/joni-2.1.2.jar:/opt/hbase/bin/= ../lib/hadoop-yarn-client-2.5.1.jar:/opt/hbase/bin/../lib/jaxb-api-2.2.2.ja= r:/opt/hbase/bin/../lib/hadoop-yarn-api-2.5.1.jar:/opt/hbase/bin/../lib/com= mons-digester-1.8.jar:/opt/hbase/bin/../lib/jersey-core-1.9.jar:/opt/hbase/= bin/../lib/hbase-procedure-1.2.2.jar:/opt/hbase/bin/../lib/slf4j-api-1.7.7.= jar:/opt/hbase/bin/../lib/commons-logging-1.2.jar:/opt/hbase/bin/../lib/hba= se-it-1.2.2.jar:/opt/hbase/bin/../lib/guice-3.0.jar:/opt/hbase/bin/../lib/a= pacheds-i18n-2.0.0-M15.jar:/opt/hbase/bin/../lib/findbugs-annotations-1.3.9= -1.jar:/opt/hbase/bin/../lib/api-util-1.0.0-M20.jar:/opt/hbase/bin/../lib/a= sm-3.1.jar:/opt/hbase/bin/../lib/hbase-hadoop-compat-1.2.2.jar:/opt/hbase/b= in/../lib/hadoop-mapreduce-client-jobclient-2.5.1.jar:/opt/hbase/bin/../lib= /hadoop-mapreduce-client-common-2.5.1.jar:/opt/hbase/bin/../lib/hbase-commo= n-1.2.2.jar:/opt/hbase/bin/../lib/jsp-2.1-6.1.14.jar:/opt/hbase/bin/../lib/= nekohtml-1.9.16.jar:/opt/hbase/bin/../lib/java-xmlbuilder-0.4.jar:/opt/hbas= e/bin/../lib/hbase-protocol-1.2.2.jar:/opt/hbase/bin/../lib/esapi-2.1.0.1.j= ar:/opt/hbase/bin/../lib/junit-4.12.jar:/opt/hbase/bin/../lib/commons-fileu= pload-1.3.1.jar:/opt/hbase/bin/../lib/jamon-runtime-2.4.1.jar:/opt/hbase/bi= n/../lib/servlet-api-2.5-6.1.14.jar:/opt/hbase/bin/../lib/hbase-it-1.2.2-te= sts.jar:/opt/hbase/bin/../lib/slf4j-log4j12-1.7.5.jar:/opt/hbase/bin/../lib= /xom-1.2.5.jar:/opt/hbase/bin/../lib/avro-1.7.4.jar:/opt/hbase/bin/../lib/h= adoop-common-2.5.1.jar:/opt/hbase/bin/../lib/leveldbjni-all-1.8.jar:/opt/hb= ase/bin/../lib/commons-io-2.4.jar:/opt/hbase/bin/../lib/batik-css-1.8.jar:/= opt/hbase/bin/../lib/commons-codec-1.9.jar:/opt/hbase/bin/../lib/jaxb-impl-= 2.2.3-1.jar:/opt/hbase/bin/../lib/hadoop-yarn-common-2.5.1.jar:/opt/hbase/b= in/../lib/javax.inject-1.jar:/opt/hbase/bin/../lib/commons-httpclient-3.1.j= ar:/opt/hbase/bin/../lib/commons-configuration-1.6.jar:/opt/hbase/bin/../li= b/zookeeper-3.4.6.jar:/opt/hbase/bin/../lib/jets3t-0.9.0.jar:/opt/hbase/bin= /../lib/guice-servlet-3.0.jar:/opt/hbase/bin/../lib/jersey-client-1.9.jar:/= opt/hbase/bin/../lib/hbase-common-1.2.2-tests.jar:/opt/hbase/bin/../lib/jas= per-compiler-5.5.23.jar:/opt/hbase/bin/../lib/hadoop-mapreduce-client-app-2= .5.1.jar:/opt/hbase/bin/../lib/aopalliance-1.0.jar:/opt/hbase/bin/../lib/an= tisamy-1.5.3.jar:/opt/hbase/bin/../lib/jsr305-1.3.9.jar:/opt/hbase/bin/../l= ib/metrics-core-2.2.0.jar:/opt/hbase/bin/../lib/xml-apis-1.3.03.jar:/opt/hb= ase/bin/../lib/protobuf-java-2.5.0.jar:/opt/hbase/bin/../lib/commons-math-2= .2.jar:/opt/hbase/bin/../lib/jettison-1.3.3.jar:/opt/hbase/bin/../lib/netty= -all-4.0.23.Final.jar:/opt/hbase/bin/../lib/paranamer-2.3.jar:/opt/hbase/bi= n/../lib/xml-apis-ext-1.3.04.jar:/opt/hbase/bin/../lib/log4j-1.2.17.jar:/op= t/hbase/bin/../lib/jersey-json-1.9.jar:/opt/hbase/bin/../lib/jruby-complete= -1.6.8.jar:/opt/hbase/bin/../lib/hbase-prefix-tree-1.2.2.jar:/opt/hbase/bin= /../lib/commons-math3-3.1.1.jar:/opt/hbase/bin/../lib/jcodings-1.0.8.jar:/o= pt/hbase/bin/../lib/api-asn1-api-1.0.0-M20.jar:/opt/hbase/bin/../lib/jasper= -runtime-5.5.23.jar:/opt/hbase/bin/../lib/commons-lang-2.6.jar:/opt/hbase/b= in/../lib/hadoop-annotations-2.5.1.jar:/opt/hbase/bin/../lib/commons-beanut= ils-core-1.8.3.jar:/opt/hbase/bin/../lib/hbase-client-1.2.2.jar:/opt/hbase/= bin/../lib/hadoop-auth-2.5.1.jar:/opt/hbase/bin/../lib/batik-util-1.8.jar:/= opt/hbase/bin/../lib/htrace-core-3.1.0-incubating.jar:/opt/hbase/bin/../lib= /hbase-external-blockcache-1.2.2.jar:/opt/hbase/bin/../conf/:/opt/hbase/pho= enix-client/target/*:/opt/hbase/conf:/usr/lib/jvm/java-1.8-openjdk/lib/tool= s.jar:/opt/hbase:/opt/hbase/lib/activation-1.1.jar:/opt/hbase/lib/antisamy-= 1.5.3.jar:/opt/hbase/lib/aopalliance-1.0.jar:/opt/hbase/lib/apacheds-i18n-2= .0.0-M15.jar:/opt/hbase/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/hbas= e/lib/api-asn1-api-1.0.0-M20.jar:/opt/hbase/lib/api-util-1.0.0-M20.jar:/opt= /hbase/lib/asm-3.1.jar:/opt/hbase/lib/avro-1.7.4.jar:/opt/hbase/lib/batik-c= ss-1.8.jar:/opt/hbase/lib/batik-ext-1.8.jar:/opt/hbase/lib/batik-util-1.8.j= ar:/opt/hbase/lib/bsh-core-2.0b4.jar:/opt/hbase/lib/commons-beanutils-1.7.0= .jar:/opt/hbase/lib/commons-beanutils-core-1.8.3.jar:/opt/hbase/lib/commons= -cli-1.2.jar:/opt/hbase/lib/commons-codec-1.9.jar:/opt/hbase/lib/commons-co= llections-3.2.2.jar:/opt/hbase/lib/commons-compress-1.4.1.jar:/opt/hbase/li= b/commons-configuration-1.6.jar:/opt/hbase/lib/commons-daemon-1.0.13.jar:/o= pt/hbase/lib/commons-digester-1.8.jar:/opt/hbase/lib/commons-el-1.0.jar:/op= t/hbase/lib/commons-fileupload-1.3.1.jar:/opt/hbase/lib/commons-httpclient-= 3.1.jar:/opt/hbase/lib/commons-io-2.4.jar:/opt/hbase/lib/commons-lang-2.6.j= ar:/opt/hbase/lib/commons-logging-1.2.jar:/opt/hbase/lib/commons-math-2.2.j= ar:/opt/hbase/lib/commons-math3-3.1.1.jar:/opt/hbase/lib/commons-net-3.1.ja= r:/opt/hbase/lib/disruptor-3.3.0.jar:/opt/hbase/lib/esapi-2.1.0.1.jar:/opt/= hbase/lib/findbugs-annotations-1.3.9-1.jar:/opt/hbase/lib/guava-12.0.1.jar:= /opt/hbase/lib/guice-3.0.jar:/opt/hbase/lib/guice-servlet-3.0.jar:/opt/hbas= e/lib/hadoop-annotations-2.5.1.jar:/opt/hbase/lib/hadoop-auth-2.5.1.jar:/op= t/hbase/lib/hadoop-client-2.5.1.jar:/opt/hbase/lib/hadoop-common-2.5.1.jar:= /opt/hbase/lib/hadoop-hdfs-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client= -app-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-common-2.5.1.jar:/opt= /hbase/lib/hadoop-mapreduce-client-core-2.5.1.jar:/opt/hbase/lib/hadoop-map= reduce-client-jobclient-2.5.1.jar:/opt/hbase/lib/hadoop-mapreduce-client-sh= uffle-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-api-2.5.1.jar:/opt/hbase/lib/had= oop-yarn-client-2.5.1.jar:/opt/hbase/lib/hadoop-yarn-common-2.5.1.jar:/opt/= hbase/lib/hadoop-yarn-server-common-2.5.1.jar:/opt/hbase/lib/hbase-annotati= ons-1.2.2-tests.jar:/opt/hbase/lib/hbase-annotations-1.2.2.jar:/opt/hbase/l= ib/hbase-client-1.2.2.jar:/opt/hbase/lib/hbase-common-1.2.2-tests.jar:/opt/= hbase/lib/hbase-common-1.2.2.jar:/opt/hbase/lib/hbase-examples-1.2.2.jar:/o= pt/hbase/lib/hbase-external-blockcache-1.2.2.jar:/opt/hbase/lib/hbase-hadoo= p-compat-1.2.2.jar:/opt/hbase/lib/hbase-hadoop2-compat-1.2.2.jar:/opt/hbase= /lib/hbase-it-1.2.2-tests.jar:/opt/hbase/lib/hbase-it-1.2.2.jar:/opt/hbase/= lib/hbase-prefix-tree-1.2.2.jar:/opt/hbase/lib/hbase-procedure-1.2.2.jar:/o= pt/hbase/lib/hbase-protocol-1.2.2.jar:/opt/hbase/lib/hbase-resource-bundle-= 1.2.2.jar:/opt/hbase/lib/hbase-rest-1.2.2.jar:/opt/hbase/lib/hbase-server-1= .2.2-tests.jar:/opt/hbase/lib/hbase-server-1.2.2.jar:/opt/hbase/lib/hbase-s= hell-1.2.2.jar:/opt/hbase/lib/hbase-thrift-1.2.2.jar:/opt/hbase/lib/htrace-= core-3.1.0-incubating.jar:/opt/hbase/lib/httpclient-4.2.5.jar:/opt/hbase/li= b/httpcore-4.4.1.jar:/opt/hbase/lib/jackson-core-asl-1.9.13.jar:/opt/hbase/= lib/jackson-jaxrs-1.9.13.jar:/opt/hbase/lib/jackson-mapper-asl-1.9.13.jar:/= opt/hbase/lib/jackson-xc-1.9.13.jar:/opt/hbase/lib/jamon-runtime-2.4.1.jar:= /opt/hbase/lib/jasper-compiler-5.5.23.jar:/opt/hbase/lib/jasper-runtime-5.5= .23.jar:/opt/hbase/lib/java-xmlbuilder-0.4.jar:/opt/hbase/lib/javax.inject-= 1.jar:/opt/hbase/lib/jaxb-api-2.2.2.jar:/opt/hbase/lib/jaxb-impl-2.2.3-1.ja= r:/opt/hbase/lib/jcodings-1.0.8.jar:/opt/hbase/lib/jersey-client-1.9.jar:/o= pt/hbase/lib/jersey-core-1.9.jar:/opt/hbase/lib/jersey-guice-1.9.jar:/opt/h= base/lib/jersey-json-1.9.jar:/opt/hbase/lib/jersey-server-1.9.jar:/opt/hbas= e/lib/jets3t-0.9.0.jar:/opt/hbase/lib/jettison-1.3.3.jar:/opt/hbase/lib/jet= ty-6.1.26.jar:/opt/hbase/lib/jetty-sslengine-6.1.26.jar:/opt/hbase/lib/jett= y-util-6.1.26.jar:/opt/hbase/lib/joni-2.1.2.jar:/opt/hbase/lib/jruby-comple= te-1.6.8.jar:/opt/hbase/lib/jsch-0.1.42.jar:/opt/hbase/lib/jsp-2.1-6.1.14.j= ar:/opt/hbase/lib/jsp-api-2.1-6.1.14.jar:/opt/hbase/lib/jsr305-1.3.9.jar:/o= pt/hbase/lib/junit-4.12.jar:/opt/hbase/lib/leveldbjni-all-1.8.jar:/opt/hbas= e/lib/libthrift-0.9.3.jar:/opt/hbase/lib/log4j-1.2.17.jar:/opt/hbase/lib/me= trics-core-2.2.0.jar:/opt/hbase/lib/nekohtml-1.9.16.jar:/opt/hbase/lib/nett= y-all-4.0.23.Final.jar:/opt/hbase/lib/paranamer-2.3.jar:/opt/hbase/lib/phoe= nix-4.8.0-HBase-1.2-client.jar:/opt/hbase/lib/phoenix-4.8.0-HBase-1.2-serve= r.jar:/opt/hbase/lib/protobuf-java-2.5.0.jar:/opt/hbase/lib/servlet-api-2.5= -6.1.14.jar:/opt/hbase/lib/servlet-api-2.5.jar:/opt/hbase/lib/slf4j-api-1.7= .7.jar:/opt/hbase/lib/slf4j-log4j12-1.7.5.jar:/opt/hbase/lib/snappy-java-1.= 0.4.1.jar:/opt/hbase/lib/spymemcached-2.11.6.jar:/opt/hbase/lib/xalan-2.7.0= .jar:/opt/hbase/lib/xml-apis-1.3.03.jar:/opt/hbase/lib/xml-apis-ext-1.3.04.= jar:/opt/hbase/lib/xmlenc-0.52.jar:/opt/hbase/lib/xom-1.2.5.jar:/opt/hbase/= lib/xz-1.0.jar:/opt/hbase/lib/zookeeper-3.4.6.jar:: > 2016-09-02 00:26:51,572 INFO [main] zookeeper.ZooKeeper: Client environm= ent:java.library.path=3D/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:= /usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/j= re/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/li= b > 2016-09-02 00:26:51,572 INFO [main] zookeeper.ZooKeeper: Client environm= ent:java.io.tmpdir=3D/tmp > 2016-09-02 00:26:51,572 INFO [main] zookeeper.ZooKeeper: Client environm= ent:java.compiler=3D > 2016-09-02 00:26:51,572 INFO [main] zookeeper.ZooKeeper: Client environm= ent:os.name=3DLinux > 2016-09-02 00:26:51,572 INFO [main] zookeeper.ZooKeeper: Client environm= ent:os.arch=3Damd64 > 2016-09-02 00:26:51,572 INFO [main] zookeeper.ZooKeeper: Client environm= ent:os.version=3D4.4.0-36-generic > 2016-09-02 00:26:51,572 INFO [main] zookeeper.ZooKeeper: Client environm= ent:user.name=3Dhbase > 2016-09-02 00:26:51,572 INFO [main] zookeeper.ZooKeeper: Client environm= ent:user.home=3D/opt/hbase > 2016-09-02 00:26:51,572 INFO [main] zookeeper.ZooKeeper: Client environm= ent:user.dir=3D/opt/hbase > 2016-09-02 00:26:51,573 INFO [main] zookeeper.ZooKeeper: Initiating clie= nt connection, connectString=3Dm9edd51-zookeeper.m9edd51 sessionTimeout=3D9= 0000 watcher=3Dorg.apache.tephra.zookeeper.TephraZKClientService$5@45c7e403 > 2016-09-02 00:26:51,596 INFO [main-SendThread(m9edd51-zookeeper.m9edd51:= 2181)] zookeeper.ClientCnxn: Opening socket connection to server m9edd51-zo= okeeper.m9edd51/172.18.0.2:2181. Will not attempt to authenticate using SAS= L (unknown error) > 2016-09-02 00:26:51,600 INFO [main-SendThread(m9edd51-zookeeper.m9edd51:= 2181)] zookeeper.ClientCnxn: Socket connection established to m9edd51-zooke= eper.m9edd51/172.18.0.2:2181, initiating session > 2016-09-02 00:26:51,605 INFO [main-SendThread(m9edd51-zookeeper.m9edd51:= 2181)] zookeeper.ClientCnxn: Session establishment complete on server m9edd= 51-zookeeper.m9edd51/172.18.0.2:2181, sessionid =3D 0x156e849e51c0007, nego= tiated timeout =3D 40000 > 2016-09-02 00:26:51,625 INFO [main] inmemory.InMemoryTransactionService:= Configuring TransactionService, address: 0.0.0.0, port: 15165, threads: 20= , io threads: 2, max read buffer (bytes): 16777216 > 2016-09-02 00:26:51,626 INFO [main] tephra.TransactionServiceMain: Start= ing TransactionServiceMain > 2016-09-02 00:26:51,633 INFO [main] zookeeper.LeaderElection: Start lead= er election on m9edd51-zookeeper.m9edd51/tx.service/leader with guid aa0cbc= c9-5267-4618-9059-bb90a1e7e948 > 2016-09-02 00:26:51,694 INFO [leader-election-tx.service-leader] metrics= .DefaultMetricsCollector: Configured metrics report to emit every 60 second= s > 2016-09-02 00:26:51,863 INFO [ThriftRPCServer] tephra.TransactionManager= : Starting transaction manager. > 2016-09-02 00:26:51,874 INFO [DefaultMetricsCollector STARTING] metrics.= DefaultMetricsCollector: Started metrics reporter > 2016-09-02 00:26:51,954 WARN [HDFSTransactionStateStorage STARTING] util= .NativeCodeLoader: Unable to load native-hadoop library for your platform..= . using builtin-java classes where applicable > 2016-09-02 00:26:52,554 INFO [HDFSTransactionStateStorage STARTING] pers= ist.HDFSTransactionStateStorage: Using snapshot dir /tmp/tephra/snapshots > 2016-09-02 00:26:52,594 INFO [ThriftRPCServer] persist.HDFSTransactionSt= ateStorage: Creating snapshot dir at /tmp/tephra/snapshots > 2016-09-02 00:26:52,620 INFO [ThriftRPCServer] persist.HDFSTransactionSt= ateStorage: No snapshot files found in /tmp/tephra/snapshots > 2016-09-02 00:26:52,624 INFO [ThriftRPCServer] tephra.TransactionManager= : Starting periodic timed-out transaction cleanup every 10 seconds with def= ault timeout of 60 seconds. > 2016-09-02 00:26:52,624 INFO [ThriftRPCServer] tephra.TransactionManager= : Starting periodic snapshot thread, frequency =3D 300 seconds, location = =3D /tmp/tephra/snapshots > 2016-09-02 00:26:52,624 INFO [ThriftRPCServer] tephra.TransactionManager= : Starting periodic Metrics Emitter thread, frequency =3D 10000 > 2016-09-02 00:26:52,631 INFO [ThriftRPCServer] rpc.ThriftRPCServer: Star= ting RPC server for TTransactionServer > 2016-09-02 00:26:52,641 INFO [ThriftRPCServer] rpc.ThriftRPCServer: Runn= ing RPC server for TTransactionServer > 2016-09-02 00:26:52,641 INFO [ThriftRPCServer] server.TThreadedSelectorS= erverWithFix: Starting TThreadedSelectorServerWithFix > 2016-09-02 00:26:52,720 INFO [leader-election-tx.service-leader] distrib= uted.TransactionService: Transaction Thrift Service started successfully on= m9edd51-hmaster1.m9edd51/172.18.0.12:15165 > 2016-09-02 00:27:38,642 INFO [main-SendThread(m9edd51-zookeeper.m9edd51:= 2181)] zookeeper.ClientCnxn: Unable to read additional data from server ses= sionid 0x156e849e51c0007, likely server has closed socket, closing socket c= onnection and attempting reconnect > 2016-09-02 00:27:38,746 INFO [leader-election-tx.service-leader] zookeep= er.LeaderElection: Disconnected from ZK: m9edd51-zookeeper.m9edd51 for /tx.= service/leader > 2016-09-02 00:27:38,746 INFO [leader-election-tx.service-leader] zookeep= er.LeaderElection: Stepping down as leader due to disconnect: m9edd51-zooke= eper.m9edd51 for /tx.service/leader > 2016-09-02 00:27:38,747 INFO [leader-election-tx.service-leader] rpc.Thr= iftRPCServer: Request to stop RPC server for TTransactionServer > 2016-09-02 00:27:38,751 INFO [ThriftRPCServer] rpc.ThriftRPCServer: Done= running RPC server for TTransactionServer > 2016-09-02 00:27:38,751 INFO [ThriftRPCServer] tephra.TransactionManager= : Shutting down gracefully... > 2016-09-02 00:27:38,752 INFO [tx-clean-timeout] tephra.TransactionManage= r: Exiting thread tx-clean-timeout > 2016-09-02 00:27:38,752 INFO [tx-metrics] tephra.TransactionManager: Exi= ting thread tx-metrics > 2016-09-02 00:27:38,752 INFO [tx-snapshot] tephra.TransactionManager: Wr= iting final snapshot prior to shutdown > 2016-09-02 00:27:38,884 INFO [tx-snapshot] tephra.TransactionManager: Ex= iting thread tx-snapshot > 2016-09-02 00:27:38,992 INFO [DefaultMetricsCollector STOPPING] metrics.= DefaultMetricsCollector: Stopped metrics reporter > 2016-09-02 00:27:38,992 INFO [ThriftRPCServer] tephra.TransactionManager= : Took 240.2 ms to stop > 2016-09-02 00:27:38,993 INFO [ThriftRPCServer] rpc.ThriftRPCServer: RPC = server for TTransactionServer stopped. > 2016-09-02 00:27:39,013 INFO [main-SendThread(m9edd51-zookeeper.m9edd51:= 2181)] zookeeper.ClientCnxn: Opening socket connection to server m9edd51-zo= okeeper.m9edd51/fe80:0:0:0:0:242:ac12:2:2181. Will not attempt to authentic= ate using SASL (unknown error) > 2016-09-02 00:27:39,014 WARN [main-SendThread(m9edd51-zookeeper.m9edd51:= 2181)] zookeeper.ClientCnxn: Session 0x156e849e51c0007 for server null, une= xpected error, closing socket connection and attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.j= ava:717) > at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnx= nSocketNIO.java:361) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java= :1081) > 2016-09-02 00:27:40,462 INFO [main-SendThread(m9edd51-zookeeper.m9edd51:= 2181)] zookeeper.ClientCnxn: Opening socket connection to server m9edd51-zo= okeeper.m9edd51/172.18.0.2:2181. Will not attempt to authenticate using SAS= L (unknown error) > 2016-09-02 00:27:40,463 INFO [main-SendThread(m9edd51-zookeeper.m9edd51:= 2181)] zookeeper.ClientCnxn: Socket connection established to m9edd51-zooke= eper.m9edd51/172.18.0.2:2181, initiating session > 2016-09-02 00:27:40,464 INFO [main-SendThread(m9edd51-zookeeper.m9edd51:= 2181)] zookeeper.ClientCnxn: Session establishment complete on server m9edd= 51-zookeeper.m9edd51/172.18.0.2:2181, sessionid =3D 0x156e849e51c0007, nego= tiated timeout =3D 40000 > 2016-09-02 00:27:40,469 INFO [leader-election-tx.service-leader] zookeep= er.LeaderElection: Connected to ZK, running election: m9edd51-zookeeper.m9e= dd51 for /tx.service/leader > 2016-09-02 00:27:40,502 INFO [ThriftRPCServer] rpc.ThriftRPCServer: Star= ting RPC server for TTransactionServer > 2016-09-02 00:27:40,502 INFO [ThriftRPCServer] rpc.ThriftRPCServer: Runn= ing RPC server for TTransactionServer > 2016-09-02 00:27:40,502 INFO [ThriftRPCServer] server.TThreadedSelectorS= erverWithFix: Starting TThreadedSelectorServerWithFix > 2016-09-02 00:27:40,509 INFO [leader-election-tx.service-leader] distrib= uted.TransactionService: Transaction Thrift Service started successfully on= m9edd51-hmaster1.m9edd51/172.18.0.12:15165 > 2016-09-02 00:28:52,892 ERROR [TTransactionServer-rpc-0] thrift.ProcessFu= nction: Internal error processing startShort > java.lang.IllegalStateException: Transaction Manager is not running. > at com.google.common.base.Preconditions.checkState(Preconditions.= java:149) > at org.apache.tephra.TransactionManager.ensureAvailable(Transacti= onManager.java:709) > at org.apache.tephra.TransactionManager.startTx(TransactionManage= r.java:768) > at org.apache.tephra.TransactionManager.startShort(TransactionMan= ager.java:728) > at org.apache.tephra.TransactionManager.startShort(TransactionMan= ager.java:716) > at org.apache.tephra.distributed.TransactionServiceThriftHandler.= startShort(TransactionServiceThriftHandler.java:71) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth= odAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.tephra.rpc.ThriftRPCServer$1.invoke(ThriftRPCServer= .java:261) > at com.sun.proxy.$Proxy17.startShort(Unknown Source) > at org.apache.tephra.distributed.thrift.TTransactionServer$Proces= sor$startShort.getResult(TTransactionServer.java:974) > at org.apache.tephra.distributed.thrift.TTransactionServer$Proces= sor$startShort.getResult(TTransactionServer.java:959) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java= :39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:3= 9) > at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer= .invoke(AbstractNonblockingServer.java:478) > at org.apache.thrift.server.Invocation.run(Invocation.java:18) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > 2016-09-02 00:28:53,297 ERROR [TTransactionServer-rpc-0] thrift.ProcessFu= nction: Internal error processing startShort > java.lang.IllegalStateException: Transaction Manager is not running. > at com.google.common.base.Preconditions.checkState(Preconditions.= java:149) > at org.apache.tephra.TransactionManager.ensureAvailable(Transacti= onManager.java:709) > at org.apache.tephra.TransactionManager.startTx(TransactionManage= r.java:768) > at org.apache.tephra.TransactionManager.startShort(TransactionMan= ager.java:728) > at org.apache.tephra.TransactionManager.startShort(TransactionMan= ager.java:716) > at org.apache.tephra.distributed.TransactionServiceThriftHandler.= startShort(TransactionServiceThriftHandler.java:71) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth= odAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.tephra.rpc.ThriftRPCServer$1.invoke(ThriftRPCServer= .java:261) > at com.sun.proxy.$Proxy17.startShort(Unknown Source) > at org.apache.tephra.distributed.thrift.TTransactionServer$Proces= sor$startShort.getResult(TTransactionServer.java:974) > at org.apache.tephra.distributed.thrift.TTransactionServer$Proces= sor$startShort.getResult(TTransactionServer.java:959) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java= :39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:3= 9) > at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer= .invoke(AbstractNonblockingServer.java:478) > at org.apache.thrift.server.Invocation.run(Invocation.java:18) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > 2016-09-02 00:28:54,901 ERROR [TTransactionServer-rpc-0] thrift.ProcessFu= nction: Internal error processing startShort > java.lang.IllegalStateException: Transaction Manager is not running. > at com.google.common.base.Preconditions.checkState(Preconditions.= java:149) > at org.apache.tephra.TransactionManager.ensureAvailable(Transacti= onManager.java:709) > at org.apache.tephra.TransactionManager.startTx(TransactionManage= r.java:768) > at org.apache.tephra.TransactionManager.startShort(TransactionMan= ager.java:728) > at org.apache.tephra.TransactionManager.startShort(TransactionMan= ager.java:716) > at org.apache.tephra.distributed.TransactionServiceThriftHandler.= startShort(TransactionServiceThriftHandler.java:71) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth= odAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.tephra.rpc.ThriftRPCServer$1.invoke(ThriftRPCServer= .java:261) > at com.sun.proxy.$Proxy17.startShort(Unknown Source) > at org.apache.tephra.distributed.thrift.TTransactionServer$Proces= sor$startShort.getResult(TTransactionServer.java:974) > at org.apache.tephra.distributed.thrift.TTransactionServer$Proces= sor$startShort.getResult(TTransactionServer.java:959) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java= :39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:3= 9) > at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer= .invoke(AbstractNonblockingServer.java:478) > at org.apache.thrift.server.Invocation.run(Invocation.java:18) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > 2016-09-02 00:29:01,313 ERROR [TTransactionServer-rpc-0] thrift.ProcessFu= nction: Internal error processing startShort > java.lang.IllegalStateException: Transaction Manager is not running. > at com.google.common.base.Preconditions.checkState(Preconditions.= java:149) > at org.apache.tephra.TransactionManager.ensureAvailable(Transacti= onManager.java:709) > at org.apache.tephra.TransactionManager.startTx(TransactionManage= r.java:768) > at org.apache.tephra.TransactionManager.startShort(TransactionMan= ager.java:728) > at org.apache.tephra.TransactionManager.startShort(TransactionMan= ager.java:716) > at org.apache.tephra.distributed.TransactionServiceThriftHandler.= startShort(TransactionServiceThriftHandler.java:71) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth= odAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.tephra.rpc.ThriftRPCServer$1.invoke(ThriftRPCServer= .java:261) > at com.sun.proxy.$Proxy17.startShort(Unknown Source) > at org.apache.tephra.distributed.thrift.TTransactionServer$Proces= sor$startShort.getResult(TTransactionServer.java:974) > at org.apache.tephra.distributed.thrift.TTransactionServer$Proces= sor$startShort.getResult(TTransactionServer.java:959) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java= :39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:3= 9) > at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer= .invoke(AbstractNonblockingServer.java:478) > at org.apache.thrift.server.Invocation.run(Invocation.java:18) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {code} > Although it thinks that the transaction manager is not running, I can sti= ll see that it is running using jps: > {code} > bash-4.3# jps > 138 HMaster > 974 Jps > 191 TransactionServiceMain > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)