From users-return-8649-archive-asf-public=cust-asf.ponee.io@zeppelin.apache.org Thu Sep 5 20:25:43 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 85EF9180656 for ; Thu, 5 Sep 2019 22:25:42 +0200 (CEST) Received: (qmail 44485 invoked by uid 500); 6 Sep 2019 05:32:38 -0000 Mailing-List: contact users-help@zeppelin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@zeppelin.apache.org Delivered-To: mailing list users@zeppelin.apache.org Received: (qmail 44475 invoked by uid 99); 6 Sep 2019 05:32:37 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Sep 2019 05:32:37 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 519FA180F9C for ; Thu, 5 Sep 2019 20:25:40 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.811 X-Spam-Level: * X-Spam-Status: No, score=1.811 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=placeiq.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id mObWSC3WznUr for ; Thu, 5 Sep 2019 20:25:36 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::735; helo=mail-qk1-x735.google.com; envelope-from=pbrenner@placeiq.com; receiver= Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id C11237DD32 for ; Thu, 5 Sep 2019 20:25:35 +0000 (UTC) Received: by mail-qk1-x735.google.com with SMTP id i78so3465679qke.11 for ; Thu, 05 Sep 2019 13:25:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=placeiq.com; s=google; h=date:from:to:message-id:in-reply-to:references:subject:mime-version; bh=tHha5Usn8jtcgJE+We6ZV5jeZ8oKk0F6UV2gFU4BOqI=; b=ebIcDmdsnuxxEqP3lkrDy7+1tRR7Rq/qB8Kay53n9NhPu9J/QDwSFpgnwec7sLgBy3 lvYJYEO+fCqIOn89dGi+W/AyVAYmPxuCJq+vMGGYF1GjtQeqmHYz/0sAyd+T036+x1tl LPuKHD2TxMyOKKBXBUmzbnrDRb3JxuXDCPnI0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:message-id:in-reply-to:references :subject:mime-version; bh=tHha5Usn8jtcgJE+We6ZV5jeZ8oKk0F6UV2gFU4BOqI=; b=WbKIyv6BCqY5CkU4lb/VNMAEufrnpLLnomJk5sKKSr1e6I2bYxY2YbsNpwBSnBh2Gu XzY3SRUD15D/BzL0FeomQQEYH4U5FSElt3iVhTSpWzyT2Z/d6q0LeMRIC/HBWVzvbOv/ +wahAn9ZOvNJcwArf6Y54Ws+W0lhLdz4/gn1NWSzcyrD6EYp65UethlL1kD1PXH+jEy3 XcKyxVLw/3Z4HqR3CQD6Lcm35j8Z2gmNvdvnZjdW/9hldWTZCbVP2uPV75mz86A/JaZU 3nHy59mrQ9GdvYng6ttJjnI4t9kfdzNp1v8yg6Vi7/YWwZMBQHL+eUFD5XchbgsKuFUk EPIQ== X-Gm-Message-State: APjAAAVviKyS+AlblddMQERD8I1yt5UnJQKT1adCncTFqsam0zFUGP07 GlFbMp7amkzGSXuwxya3kN6tN8zoCOMyug== X-Google-Smtp-Source: APXvYqxzVMnBLBT3W2l3AgeB6+OZ2gjZxuXZ73uTp16v6bK3EKjduWdD2hMam4nDGRq0DVF9jMFGvQ== X-Received: by 2002:ae9:dec2:: with SMTP id s185mr5149876qkf.203.1567715128140; Thu, 05 Sep 2019 13:25:28 -0700 (PDT) Received: from [10.201.12.49] ([204.145.86.152]) by smtp.gmail.com with ESMTPSA id z123sm1699034qke.96.2019.09.05.13.25.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 05 Sep 2019 13:25:27 -0700 (PDT) Date: Thu, 5 Sep 2019 16:25:15 -0400 From: Paul Brenner To: users@zeppelin.apache.org Message-ID: <69783a67-4127-48f2-9e1c-7da30cfae1b9@Spark> In-Reply-To: References: <2dae7ce3-c9e2-4d20-b8cd-258439032219@Spark> <094e70cb-9c94-4c4d-9997-f47a14315feb@Spark> Subject: Re: Any way to diagnose TTransportException errors? X-Readdle-Message-ID: 69783a67-4127-48f2-9e1c-7da30cfae1b9@Spark MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="5d716f36_17a1b582_12284" --5d716f36_17a1b582_12284 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Thanks all, I will try 0.9 and report back. PAUL BRENNER Head of Data Science pbrenner=40placeiq.com =7C (217) 390-3033 =7C www.placeiq.com twitter =40placeiq linkedin /placeiq On Sep 5, 2019, 5:52 AM -0400, James Srinivasan , wrote: > I had to use 0.9.1 due to using Accumulo 1.7 (which requires 0.9.1) in = some extra JARs I wanted to use with Spark. In the end, I built Zeppelin = specifying 0.9.1 and it worked for me. > > > On Wed, 4 Sep 2019 at 16:38, Paul Brenner wr= ote: > > > It looks like we do have two versions: > > > > > > ./zeppelin/spark/spark-dependencies/target/spark-2.4.0/pom.xml: 0.9.3 > > > > > > ./zeppelin/pom.xml: 0.9.2 > > > > > > I=E2=80=99m always hesitant to rebuild blindly with changes. Did yo= u align your versions to 0.9.3 and have success=3F Can anyone weigh in on= this discrepancy=3F > > > 11:33 AM > > > ./zeppelin/pom.xml:=C2=A0=C2=A0=C2=A0=C2=A00.9.2= > > > > > > > > > > > > PAUL BRENNER > > > Head of Data Science > > > pbrenner=40placeiq.com =7C (217) 390-3033 =7C www.placeiq.com > > > twitter =40placeiq linkedin /placeiq > > > On Sep 4, 2019, 7:57 AM -0400, James Srinivasan , wrote: > > > > Is there a chance you have differing versions of libthrift on you= r classpath=3F I've seen errors (admittedly not looking like yours) for t= hat reason=3F > > > > > > > > James > > > > > > > > > On Sun, 1 Sep 2019 at 00:57, Paul Brenner wrote: > > > > > > We are using a recent build post 0.81 so 0.82=3F > > > > > > > > > > > > Restarting the interpreter helps in 75% of the cases, but the= remaining few just won=E2=80=99t recover until we shut everything down a= nd restart all if zeppelin. I suspect there is some other process that we= aren=E2=80=99t identifying that needs killing. > > > > > > > > > > > > > > > > > > > > > > > > PAUL BRENNER > > > > > > Head of Data Science > > > > > > pbrenner=40placeiq.com =7C (217) 390-3033 =7C www.placeiq.com= > > > > > > twitter =40placeiq linkedin /placeiq > > > > > > On Aug 31, 2019, 7:50 PM -0400, Jeff Zhang , wrote: > > > > > > > Which version do you use =3F And does restarting interprete= r work for you =3F > > > > > > > > > > > > > > > Paul Brenner =E4=BA=8E2019=E5=B9= =B48=E6=9C=8830=E6=97=A5=E5=91=A8=E4=BA=94 =E4=B8=8B=E5=8D=8811:45=E5=86=99= =E9=81=93=EF=BC=9A > > > > > > > > > We run into the error pasted below semi-regularly and o= ften the only solution is to do a full restart of zeppelin to get the use= r/notebook back up and running. I=E2=80=99m at a loss of where to even lo= ok to try to debug this or address it. Anyone have any ideas or any way t= o point me in the right direction=3F > > > > > > > > > > > > > > > > > > This is what the notebook shows: > > > > > > > > > org.apache.thrift.transport.TTransportException at org.= apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:1= 32) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)= at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.ja= va:429) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProt= ocol.java:318) at org.apache.thrift.protocol.TBinaryProtocol.readMessageB= egin(TBinaryProtocol.java:219) at org.apache.thrift.TServiceClient.receiv= eBase(TServiceClient.java:69) at org.apache.zeppelin.interpreter.thrift.R= emoteInterpreterService=24Client.recv=5Finterpret(RemoteInterpreterServic= e.java:274) at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterSe= rvice=24Client.interpret(RemoteInterpreterService.java:258) at org.apache= .zeppelin.interpreter.remote.RemoteInterpreter=244.call(RemoteInterpreter= .java:233) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter=24= 4.call(RemoteInterpreter.java:229) at org.apache.zeppelin.interpreter.rem= ote.RemoteInterpreterProcess.callRemote=46unction(RemoteInterpreterProces= s.java:135) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.i= nterpret(RemoteInterpreter.java:228) at org.apache.zeppelin.notebook.Para= graph.jobRun(Paragraph.java:437) at org.apache.zeppelin.scheduler.Job.run= (Job.java:188) at org.apache.zeppelin.scheduler.RemoteScheduler=24JobRunn= er.run(RemoteScheduler.java:315) at java.util.concurrent.Executors=24Runn= ableAdapter.call(Executors.java:511) at java.util.concurrent.=46utureTask= .run(=46utureTask.java:266) at java.util.concurrent.ScheduledThreadPoolEx= ecutor=24Scheduled=46utureTask.access=24201(ScheduledThreadPoolExecutor.j= ava:180) at java.util.concurrent.ScheduledThreadPoolExecutor=24Scheduled=46= utureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurre= nt.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.uti= l.concurrent.ThreadPoolExecutor=24Worker.run(ThreadPoolExecutor.java:624)= at java.lang.Thread.run(Thread.java:748) > > > > > > > > > > > > > > > > > > > > > > > > > > > This is what I pulled from the log: > > > > > > > > > > > > > > > > > > ERROR =5B2019-08-30 15:41:56,923=5D (=7Bpool-2-thread-2= 4=7D Job.java=5Brun=5D:190) - Job failed > > > > > > > > > java.lang.RuntimeException: org.apache.thrift.transport= .TTransportException > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eterProcess.callRemote=46unction(RemoteInterpreterProcess.java:139) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eter.interpret(RemoteInterpreter.java:228) > > > > > > > > > at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragr= aph.java:437) > > > > > > > > > at org.apache.zeppelin.scheduler.Job.run(Job.java:188) > > > > > > > > > at org.apache.zeppelin.scheduler.RemoteScheduler=24JobR= unner.run(RemoteScheduler.java:315) > > > > > > > > > at java.util.concurrent.Executors=24RunnableAdapter.cal= l(Executors.java:511) > > > > > > > > > at java.util.concurrent.=46utureTask.run(=46utureTask.j= ava:266) > > > > > > > > > at java.util.concurrent.ScheduledThreadPoolExecutor=24S= cheduled=46utureTask.access=24201(ScheduledThreadPoolExecutor.java:180) > > > > > > > > > at java.util.concurrent.ScheduledThreadPoolExecutor=24S= cheduled=46utureTask.run(ScheduledThreadPoolExecutor.java:293) > > > > > > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker(Th= readPoolExecutor.java:1149) > > > > > > > > > at java.util.concurrent.ThreadPoolExecutor=24Worker.run= (ThreadPoolExecutor.java:624) > > > > > > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > > > > Caused by: org.apache.thrift.transport.TTransportExcept= ion > > > > > > > > > at org.apache.thrift.transport.TIOStreamTransport.read(= TIOStreamTransport.java:132) > > > > > > > > > at org.apache.thrift.transport.TTransport.readAll(TTran= sport.java:86) > > > > > > > > > at org.apache.thrift.protocol.TBinaryProtocol.readAll(T= BinaryProtocol.java:429) > > > > > > > > > at org.apache.thrift.protocol.TBinaryProtocol.readI32(T= BinaryProtocol.java:318) > > > > > > > > > at org.apache.thrift.protocol.TBinaryProtocol.readMessa= geBegin(TBinaryProtocol.java:219) > > > > > > > > > at org.apache.thrift.TServiceClient.receiveBase(TServic= eClient.java:69) > > > > > > > > > at org.apache.zeppelin.interpreter.thrift.RemoteInterpr= eterService=24Client.recv=5Finterpret(RemoteInterpreterService.java:274) > > > > > > > > > at org.apache.zeppelin.interpreter.thrift.RemoteInterpr= eterService=24Client.interpret(RemoteInterpreterService.java:258) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eter=244.call(RemoteInterpreter.java:233) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eter=244.call(RemoteInterpreter.java:229) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eterProcess.callRemote=46unction(RemoteInterpreterProcess.java:135) > > > > > > > > > ... 11 more > > > > > > > > > IN=46O =5B2019-08-30 15:41:56,954=5D (=7BExec Default E= xecutor=7D RemoteInterpreterManagedProcess.java=5BonProcessComplete=5D:24= 3) - Interpreter process exited 0 > > > > > > > > > DEBUG =5B2019-08-30 15:41:56,957=5D (=7Bpool-2-thread-2= 4=7D RemoteScheduler.java=5Brun=5D:332) - Job ABORT, 20190821-204527=5F14= 9556857, org.apache.thrift.transport.TTransportException > > > > > > > > > at org.apache.thrift.transport.TIOStreamTransport.read(= TIOStreamTransport.java:132) > > > > > > > > > at org.apache.thrift.transport.TTransport.readAll(TTran= sport.java:86) > > > > > > > > > at org.apache.thrift.protocol.TBinaryProtocol.readAll(T= BinaryProtocol.java:429) > > > > > > > > > at org.apache.thrift.protocol.TBinaryProtocol.readI32(T= BinaryProtocol.java:318) > > > > > > > > > at org.apache.thrift.protocol.TBinaryProtocol.readMessa= geBegin(TBinaryProtocol.java:219) > > > > > > > > > at org.apache.thrift.TServiceClient.receiveBase(TServic= eClient.java:69) > > > > > > > > > at org.apache.zeppelin.interpreter.thrift.RemoteInterpr= eterService=24Client.recv=5Finterpret(RemoteInterpreterService.java:274) > > > > > > > > > at org.apache.zeppelin.interpreter.thrift.RemoteInterpr= eterService=24Client.interpret(RemoteInterpreterService.java:258) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eter=244.call(RemoteInterpreter.java:233) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eter=244.call(RemoteInterpreter.java:229) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eterProcess.callRemote=46unction(RemoteInterpreterProcess.java:135) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eter.interpret(RemoteInterpreter.java:228) > > > > > > > > > at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragr= aph.java:437) > > > > > > > > > at org.apache.zeppelin.scheduler.Job.run(Job.java:188) > > > > > > > > > at org.apache.zeppelin.scheduler.RemoteScheduler=24JobR= unner.run(RemoteScheduler.java:315) > > > > > > > > > at java.util.concurrent.Executors=24RunnableAdapter.cal= l(Executors.java:511) > > > > > > > > > at java.util.concurrent.=46utureTask.run(=46utureTask.j= ava:266) > > > > > > > > > at java.util.concurrent.ScheduledThreadPoolExecutor=24S= cheduled=46utureTask.access=24201(ScheduledThreadPoolExecutor.java:180) > > > > > > > > > at java.util.concurrent.ScheduledThreadPoolExecutor=24S= cheduled=46utureTask.run(ScheduledThreadPoolExecutor.java:293) > > > > > > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker(Th= readPoolExecutor.java:1149) > > > > > > > > > at java.util.concurrent.ThreadPoolExecutor=24Worker.run= (ThreadPoolExecutor.java:624) > > > > > > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > > > > > > > > > > > > > ERROR =5B2019-08-30 15:41:56,959=5D (=7Bpool-2-thread-2= 4=7D NotebookServer.java=5BafterStatusChange=5D:2308) - Error > > > > > > > > > java.lang.RuntimeException: org.apache.thrift.transport= .TTransportException > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eterProcess.callRemote=46unction(RemoteInterpreterProcess.java:139) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eter.interpret(RemoteInterpreter.java:228) > > > > > > > > > at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragr= aph.java:437) > > > > > > > > > at org.apache.zeppelin.scheduler.Job.run(Job.java:188) > > > > > > > > > at org.apache.zeppelin.scheduler.RemoteScheduler=24JobR= unner.run(RemoteScheduler.java:315) > > > > > > > > > at java.util.concurrent.Executors=24RunnableAdapter.cal= l(Executors.java:511) > > > > > > > > > at java.util.concurrent.=46utureTask.run(=46utureTask.j= ava:266) > > > > > > > > > at java.util.concurrent.ScheduledThreadPoolExecutor=24S= cheduled=46utureTask.access=24201(ScheduledThreadPoolExecutor.java:180) > > > > > > > > > at java.util.concurrent.ScheduledThreadPoolExecutor=24S= cheduled=46utureTask.run(ScheduledThreadPoolExecutor.java:293) > > > > > > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker(Th= readPoolExecutor.java:1149) > > > > > > > > > at java.util.concurrent.ThreadPoolExecutor=24Worker.run= (ThreadPoolExecutor.java:624) > > > > > > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > > > > Caused by: org.apache.thrift.transport.TTransportExcept= ion > > > > > > > > > at org.apache.thrift.transport.TIOStreamTransport.read(= TIOStreamTransport.java:132) > > > > > > > > > at org.apache.thrift.transport.TTransport.readAll(TTran= sport.java:86) > > > > > > > > > at org.apache.thrift.protocol.TBinaryProtocol.readAll(T= BinaryProtocol.java:429) > > > > > > > > > at org.apache.thrift.protocol.TBinaryProtocol.readI32(T= BinaryProtocol.java:318) > > > > > > > > > at org.apache.thrift.protocol.TBinaryProtocol.readMessa= geBegin(TBinaryProtocol.java:219) > > > > > > > > > at org.apache.thrift.TServiceClient.receiveBase(TServic= eClient.java:69) > > > > > > > > > at org.apache.zeppelin.interpreter.thrift.RemoteInterpr= eterService=24Client.recv=5Finterpret(RemoteInterpreterService.java:274) > > > > > > > > > at org.apache.zeppelin.interpreter.thrift.RemoteInterpr= eterService=24Client.interpret(RemoteInterpreterService.java:258) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eter=244.call(RemoteInterpreter.java:233) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eter=244.call(RemoteInterpreter.java:229) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eterProcess.callRemote=46unction(RemoteInterpreterProcess.java:135) > > > > > > > > > ... 11 more > > > > > > > > > WARN =5B2019-08-30 15:41:56,959=5D (=7Bpool-2-thread-24= =7D NotebookServer.java=5BafterStatusChange=5D:2316) - Job 20190821-20452= 7=5F149556857 is finished, status: ERROR, exception: java.lang.RuntimeExc= eption: org.apache.thrift.transport.TTransportException, result: %text or= g.apache.thrift.transport.TTransportException > > > > > > > > > at org.apache.thrift.transport.TIOStreamTransport.read(= TIOStreamTransport.java:132) > > > > > > > > > at org.apache.thrift.transport.TTransport.readAll(TTran= sport.java:86) > > > > > > > > > at org.apache.thrift.protocol.TBinaryProtocol.readAll(T= BinaryProtocol.java:429) > > > > > > > > > at org.apache.thrift.protocol.TBinaryProtocol.readI32(T= BinaryProtocol.java:318) > > > > > > > > > at org.apache.thrift.protocol.TBinaryProtocol.readMessa= geBegin(TBinaryProtocol.java:219) > > > > > > > > > at org.apache.thrift.TServiceClient.receiveBase(TServic= eClient.java:69) > > > > > > > > > at org.apache.zeppelin.interpreter.thrift.RemoteInterpr= eterService=24Client.recv=5Finterpret(RemoteInterpreterService.java:274) > > > > > > > > > at org.apache.zeppelin.interpreter.thrift.RemoteInterpr= eterService=24Client.interpret(RemoteInterpreterService.java:258) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eter=244.call(RemoteInterpreter.java:233) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eter=244.call(RemoteInterpreter.java:229) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eterProcess.callRemote=46unction(RemoteInterpreterProcess.java:135) > > > > > > > > > at org.apache.zeppelin.interpreter.remote.RemoteInterpr= eter.interpret(RemoteInterpreter.java:228) > > > > > > > > > at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragr= aph.java:437) > > > > > > > > > at org.apache.zeppelin.scheduler.Job.run(Job.java:188) > > > > > > > > > at org.apache.zeppelin.scheduler.RemoteScheduler=24JobR= unner.run(RemoteScheduler.java:315) > > > > > > > > > at java.util.concurrent.Executors=24RunnableAdapter.cal= l(Executors.java:511) > > > > > > > > > at java.util.concurrent.=46utureTask.run(=46utureTask.j= ava:266) > > > > > > > > > at java.util.concurrent.ScheduledThreadPoolExecutor=24S= cheduled=46utureTask.access=24201(ScheduledThreadPoolExecutor.java:180) > > > > > > > > > at java.util.concurrent.ScheduledThreadPoolExecutor=24S= cheduled=46utureTask.run(ScheduledThreadPoolExecutor.java:293) > > > > > > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker(Th= readPoolExecutor.java:1149) > > > > > > > > > at java.util.concurrent.ThreadPoolExecutor=24Worker.run= (ThreadPoolExecutor.java:624) > > > > > > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > PAUL BRENNER > > > > > > > > > Head of Data Science > > > > > > > > > pbrenner=40placeiq.com =7C (217) 390-3033 =7C www.place= iq.com > > > > > > > > > twitter =40placeiq linkedin /placeiq > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best Regards > > > > > > > > > > > > > > Jeff Zhang --5d716f36_17a1b582_12284 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
Thanks all, I will try 0.9 and report back.



<= /td>

PAUL BRENNER

Head of Data Science

pbrenner=40placeiq.com =7C (217) 390-3= 033 =7C www.placeiq.com
twitter<= /span> =40placeiq linkedin /placeiq
3D=22Experian
3D=22PlaceIQ:
On Sep 5, 2019, 5:52 AM -0400, Jame= s Srinivasan <james.srinivasan=40gmail.com>, wrote:
I had to use 0.9.1 due to using Accumulo 1.7 (which = requires 0.9.1) in some extra JARs I wanted to use with Spark. In the end= , I built Zeppelin specifying 0.9.1 and it worked for me.

On Wed, 4 Sep 2019 at 16:= 38, Paul Brenner <pbrenn= er=40placeiq.com> wrote:
It looks like we do have two versions:

./zeppelin/spark/spark-dependencies/target/spark-2.= 4.0/pom.xml: <libthrift.version>0.9.3</libthrift.version>

./zeppelin/pom.xml: <libthrift.version>0.9.2&= lt;/libthrift.version>

I=E2=80=99m always hesitant to rebuild blindly with= changes. Did you align your versions to 0.9.3 and have success=3F Can an= yone weigh in on this discrepancy=3F
./zeppelin/pom.x= ml:&=23160;&=23160;&=23160;&=23160;<libthrift.version>0.9.2</lib= thrift.version>



PAU= L BRENNER

Head of Data Science

pbrenner=40placeiq.com =7C (217) 390-3033 =7C www.pla= ceiq.com
twitter= =40placeiq linkedin /placeiq
3D=22Experian=
<= a href=3D=22http://go.placeiq.com/WC01DPG0l0N3Gq000000e00=22 target=3D=22= =5Fblank=22>3D=22PlaceIQ:
On Sep 4, 2019, 7:57 AM -0400, Jame= s Srinivasan <james.srinivasan=40gmail.com>, wrote:
Is there a chance you have differing versions of lib= thrift on your classpath=3F I've seen errors (admittedly not looking like= yours) for that reason=3F

James

On Sun, 1 Sep 2019 at 00:= 57, Paul Brenner <pbrenner=40placeiq.com> wrote:
We are using a recent build post 0.81 so 0.82=3F

Restarting the interpreter helps in 75% of the case= s, but the remaining few just won=E2=80=99t recover until we shut everyth= ing down and restart all if zeppelin. I suspect there is some other proce= ss that we aren=E2=80=99t identifying that needs killing.&=23160;



PAU= L BRENNER

Head of Data Science

pbrenner=40placeiq.com =7C (217) 390-3033 =7C www.pla= ceiq.com
twitter= =40placeiq linkedin /placeiq
3D=22Experian=
<= a href=3D=22http://go.placeiq.com/WC01DPG0l0N3Gq000000e00=22 target=3D=22= =5Fblank=22>3D=22PlaceIQ:
On Aug 31, 2019, 7:50 PM -0400, Jef= f Zhang <zjffdu=40gmail.com>, wrote:
Which version do you use =3F And does restarting int= erpreter work for you =3F

Paul Brenner <pbrenner=40pl= aceiq.com> =E4=BA=8E2019=E5=B9=B48=E6=9C=8830=E6=97=A5=E5=91=A8=E4= =BA=94 =E4=B8=8B=E5=8D=8811:45=E5=86=99=E9=81=93=EF=BC=9A
We run into the error pasted below semi-regularly a= nd often the only solution is to do a full restart of zeppelin to get the= user/notebook back up and running. I=E2=80=99m at a loss of where to eve= n look to try to debug this or address it. Anyone have any ideas or any w= ay to point me in the right direction=3F

This is what the notebook shows:
org.apache.thrift.transport.TTransportExceptio= n at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTranspo= rt.java:132) at org.apache.thrift.transport.TTransport.readAll(TTransport= .java:86) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr= otocol.java:429) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TB= inaryProtocol.java:318) at org.apache.thrift.protocol.TBinaryProtocol.rea= dMessageBegin(TBinaryProtocol.java:219) at org.apache.thrift.TServiceClie= nt.receiveBase(TServiceClient.java:69) at org.apache.zeppelin.interpreter= .thrift.RemoteInterpreterService=24Client.recv=5Finterpret(RemoteInterpre= terService.java:274) at org.apache.zeppelin.interpreter.thrift.RemoteInte= rpreterService=24Client.interpret(RemoteInterpreterService.java:258) at o= rg.apache.zeppelin.interpreter.remote.RemoteInterpreter=244.call(RemoteIn= terpreter.java:233) at org.apache.zeppelin.interpreter.remote.RemoteInter= preter=244.call(RemoteInterpreter.java:229) at org.apache.zeppelin.interp= reter.remote.RemoteInterpreterProcess.callRemote=46unction(RemoteInterpre= terProcess.java:135) at org.apache.zeppelin.interpreter.remote.RemoteInte= rpreter.interpret(RemoteInterpreter.java:228) at org.apache.zeppelin.note= book.Paragraph.jobRun(Paragraph.java:437) at org.apache.zeppelin.schedule= r.Job.run(Job.java:188) at org.apache.zeppelin.scheduler.RemoteScheduler=24= JobRunner.run(RemoteScheduler.java:315) at java.util.concurrent.Executors= =24RunnableAdapter.call(Executors.java:511) at java.util.concurrent.=46ut= ureTask.run(=46utureTask.java:266) at java.util.concurrent.ScheduledThrea= dPoolExecutor=24Scheduled=46utureTask.access=24201(ScheduledThreadPoolExe= cutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor=24Sch= eduled=46utureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util= .concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at= java.util.concurrent.ThreadPoolExecutor=24Worker.run(ThreadPoolExecutor.= java:624) at java.lang.Thread.run(Thread.java:748)


This is what I pulled from the log:

ERROR =5B2019-08-30 15:41:56,923=5D (=7Bpool-2= -thread-24=7D Job.java=5Brun=5D:190) - Job failed
java.lang.RuntimeException: org.apache.thrift.= transport.TTransportException
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreterProcess.callRemote=46unction(RemoteInterpreterProcess.java:1= 39)
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreter.interpret(RemoteInterpreter.java:228)
at org.apache.zeppelin.notebook.Paragraph.jobR= un(Paragraph.java:437)
at org.apache.zeppelin.scheduler.Job.run(Job.j= ava:188)
at org.apache.zeppelin.scheduler.RemoteSchedul= er=24JobRunner.run(RemoteScheduler.java:315)
at java.util.concurrent.Executors=24RunnableAd= apter.call(Executors.java:511)
at java.util.concurrent.=46utureTask.run(=46ut= ureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExe= cutor=24Scheduled=46utureTask.access=24201(ScheduledThreadPoolExecutor.ja= va:180)
at java.util.concurrent.ScheduledThreadPoolExe= cutor=24Scheduled=46utureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.run= Worker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor=24W= orker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransp= ortException
at org.apache.thrift.transport.TIOStreamTransp= ort.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.read= All(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.= readAll(TBinaryProtocol.java:429)
at org.apache.thrift.protocol.TBinaryProtocol.= readI32(TBinaryProtocol.java:318)
at org.apache.thrift.protocol.TBinaryProtocol.= readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBas= e(TServiceClient.java:69)
at org.apache.zeppelin.interpreter.thrift.Remo= teInterpreterService=24Client.recv=5Finterpret(RemoteInterpreterService.j= ava:274)
at org.apache.zeppelin.interpreter.thrift.Remo= teInterpreterService=24Client.interpret(RemoteInterpreterService.java:258= )
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreter=244.call(RemoteInterpreter.java:233)
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreter=244.call(RemoteInterpreter.java:229)
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreterProcess.callRemote=46unction(RemoteInterpreterProcess.java:1= 35)
... 11 more
IN=46O =5B2019-08-30 15:41:56,954=5D (=7BExec = Default Executor=7D RemoteInterpreterManagedProcess.java=5BonProcessCompl= ete=5D:243) - Interpreter process exited 0
DEBUG =5B2019-08-30 15:41:56,957=5D (=7Bpool-2= -thread-24=7D RemoteScheduler.java=5Brun=5D:332) - Job ABORT, 20190821-20= 4527=5F149556857, org.apache.thrift.transport.TTransportException<= /div>
at org.apache.thrift.transport.TIOStreamTransp= ort.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.read= All(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.= readAll(TBinaryProtocol.java:429)
at org.apache.thrift.protocol.TBinaryProtocol.= readI32(TBinaryProtocol.java:318)
at org.apache.thrift.protocol.TBinaryProtocol.= readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBas= e(TServiceClient.java:69)
at org.apache.zeppelin.interpreter.thrift.Remo= teInterpreterService=24Client.recv=5Finterpret(RemoteInterpreterService.j= ava:274)
at org.apache.zeppelin.interpreter.thrift.Remo= teInterpreterService=24Client.interpret(RemoteInterpreterService.java:258= )
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreter=244.call(RemoteInterpreter.java:233)
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreter=244.call(RemoteInterpreter.java:229)
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreterProcess.callRemote=46unction(RemoteInterpreterProcess.java:1= 35)
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreter.interpret(RemoteInterpreter.java:228)
at org.apache.zeppelin.notebook.Paragraph.jobR= un(Paragraph.java:437)
at org.apache.zeppelin.scheduler.Job.run(Job.j= ava:188)
at org.apache.zeppelin.scheduler.RemoteSchedul= er=24JobRunner.run(RemoteScheduler.java:315)
at java.util.concurrent.Executors=24RunnableAd= apter.call(Executors.java:511)
at java.util.concurrent.=46utureTask.run(=46ut= ureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExe= cutor=24Scheduled=46utureTask.access=24201(ScheduledThreadPoolExecutor.ja= va:180)
at java.util.concurrent.ScheduledThreadPoolExe= cutor=24Scheduled=46utureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.run= Worker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor=24W= orker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

ERROR =5B2019-08-30 15:41:56,959=5D (=7Bpool-2= -thread-24=7D NotebookServer.java=5BafterStatusChange=5D:2308) - Error
java.lang.RuntimeException: org.apache.thrift.= transport.TTransportException
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreterProcess.callRemote=46unction(RemoteInterpreterProcess.java:1= 39)
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreter.interpret(RemoteInterpreter.java:228)
at org.apache.zeppelin.notebook.Paragraph.jobR= un(Paragraph.java:437)
at org.apache.zeppelin.scheduler.Job.run(Job.j= ava:188)
at org.apache.zeppelin.scheduler.RemoteSchedul= er=24JobRunner.run(RemoteScheduler.java:315)
at java.util.concurrent.Executors=24RunnableAd= apter.call(Executors.java:511)
at java.util.concurrent.=46utureTask.run(=46ut= ureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExe= cutor=24Scheduled=46utureTask.access=24201(ScheduledThreadPoolExecutor.ja= va:180)
at java.util.concurrent.ScheduledThreadPoolExe= cutor=24Scheduled=46utureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.run= Worker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor=24W= orker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransp= ortException
at org.apache.thrift.transport.TIOStreamTransp= ort.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.read= All(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.= readAll(TBinaryProtocol.java:429)
at org.apache.thrift.protocol.TBinaryProtocol.= readI32(TBinaryProtocol.java:318)
at org.apache.thrift.protocol.TBinaryProtocol.= readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBas= e(TServiceClient.java:69)
at org.apache.zeppelin.interpreter.thrift.Remo= teInterpreterService=24Client.recv=5Finterpret(RemoteInterpreterService.j= ava:274)
at org.apache.zeppelin.interpreter.thrift.Remo= teInterpreterService=24Client.interpret(RemoteInterpreterService.java:258= )
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreter=244.call(RemoteInterpreter.java:233)
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreter=244.call(RemoteInterpreter.java:229)
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreterProcess.callRemote=46unction(RemoteInterpreterProcess.java:1= 35)
... 11 more
WARN =5B2019-08-30 15:41:56,959=5D (=7Bpool-2-= thread-24=7D NotebookServer.java=5BafterStatusChange=5D:2316) - Job 20190= 821-204527=5F149556857 is finished, status: ERROR, exception: java.lang.R= untimeException: org.apache.thrift.transport.TTransportException, result:= %text org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransp= ort.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.read= All(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.= readAll(TBinaryProtocol.java:429)
at org.apache.thrift.protocol.TBinaryProtocol.= readI32(TBinaryProtocol.java:318)
at org.apache.thrift.protocol.TBinaryProtocol.= readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBas= e(TServiceClient.java:69)
at org.apache.zeppelin.interpreter.thrift.Remo= teInterpreterService=24Client.recv=5Finterpret(RemoteInterpreterService.j= ava:274)
at org.apache.zeppelin.interpreter.thrift.Remo= teInterpreterService=24Client.interpret(RemoteInterpreterService.java:258= )
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreter=244.call(RemoteInterpreter.java:233)
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreter=244.call(RemoteInterpreter.java:229)
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreterProcess.callRemote=46unction(RemoteInterpreterProcess.java:1= 35)
at org.apache.zeppelin.interpreter.remote.Remo= teInterpreter.interpret(RemoteInterpreter.java:228)
at org.apache.zeppelin.notebook.Paragraph.jobR= un(Paragraph.java:437)
at org.apache.zeppelin.scheduler.Job.run(Job.j= ava:188)
at org.apache.zeppelin.scheduler.RemoteSchedul= er=24JobRunner.run(RemoteScheduler.java:315)
at java.util.concurrent.Executors=24RunnableAd= apter.call(Executors.java:511)
at java.util.concurrent.=46utureTask.run(=46ut= ureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExe= cutor=24Scheduled=46utureTask.access=24201(ScheduledThreadPoolExecutor.ja= va:180)
at java.util.concurrent.ScheduledThreadPoolExe= cutor=24Scheduled=46utureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.run= Worker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor=24W= orker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)



PAU= L BRENNER

Head of Data Science

pbrenner=40placeiq.com =7C (217) 390-3033 =7C www.pla= ceiq.com
twitter= =40placeiq linkedin /placeiq
3D=22Experian=
<= a href=3D=22http://go.placeiq.com/WC01DPG0l0N3Gq000000e00=22 target=3D=22= =5Fblank=22>3D=22PlaceIQ:


--
Best Regards

Jeff Zhang
--5d716f36_17a1b582_12284--