Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D080010BFA for ; Wed, 1 Jan 2014 06:34:06 +0000 (UTC) Received: (qmail 97712 invoked by uid 500); 1 Jan 2014 06:34:03 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 97489 invoked by uid 500); 1 Jan 2014 06:34:02 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 97475 invoked by uid 99); 1 Jan 2014 06:34:01 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Jan 2014 06:34:01 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,T_ANY_PILL_PRICE,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of arshakn@gmail.com designates 209.85.215.181 as permitted sender) Received: from [209.85.215.181] (HELO mail-ea0-f181.google.com) (209.85.215.181) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Jan 2014 06:33:58 +0000 Received: by mail-ea0-f181.google.com with SMTP id m10so5842181eaj.12 for ; Tue, 31 Dec 2013 22:33:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=st1SkR2eo++V4kTQjmdapT4qM4bzYuE7OqnPZWEcKBk=; b=PsW9aKG0qcbkkM57XMEVQ2bAdAJiuuNXa0eVSA0GEwjzU9miMTXJkNQIV5dUoM+HMG QTDpDBVRYSbIktO3DVuQGRQJ0OlHWXsvg088ZeIMnNArL8040V2mbreMLfdX8yAkxhLd FpIiyyaL2+gzHx6m5pwHpxeuQVudAXMvmcOsAEJ5HqB4/kUYRv92QRcukef8DgelVgXn HuKMlSEB/dRXEnak6bYS/1ofL2Rp3T+xJBtEjOYFeFTlXfFE+Xhm+rdOscacMbCg0NSI DaJiUvbsboqUMnzNRablrvQZDwlmZ6UF8nf3EmzEjKoTNZJbzq4Rzn/NyjksE15AuFoE m2gw== MIME-Version: 1.0 X-Received: by 10.14.5.194 with SMTP id 42mr9763590eel.100.1388558016832; Tue, 31 Dec 2013 22:33:36 -0800 (PST) Received: by 10.14.194.73 with HTTP; Tue, 31 Dec 2013 22:33:36 -0800 (PST) In-Reply-To: References: <52C3A6C2.1090000@hoodel.com> Date: Tue, 31 Dec 2013 22:33:36 -0800 Message-ID: Subject: Re: slave tserver not responding From: Arshak Navruzyan To: "user@accumulo.apache.org" Content-Type: multipart/alternative; boundary=001a11c2772a57ea0604eee2dbef X-Virus-Checked: Checked by ClamAV on apache.org --001a11c2772a57ea0604eee2dbef Content-Type: text/plain; charset=ISO-8859-1 I am probably missing something really basic so I posted both the master and the slave log files: https://www.dropbox.com/sh/liv1mzuohyiv6uu/X5kx7AZJ6i Thanks again to everyone for the help! On Tue, Dec 31, 2013 at 10:20 PM, Arshak Navruzyan wrote: > disabled selinux (iptables already off) on both master and slave but > didn't make a difference unfortunately. > > > > On Tue, Dec 31, 2013 at 9:25 PM, Kurt Christensen wrote: > >> >> SELINUX disabled? IPTABLES configured? I have nothing else. >> >> Kurt >> >> ------ >> >> >> On 12/31/13 6:02 PM, Arshak Navruzyan wrote: >> >>> I configured a new instance with a master and a slave tserver. When I >>> do start-all on the master, the slave doesn't come up. I am wondering if >>> it's because I left the instance secret as the default. (I get an exception >>> when I try to change that). >>> >>> This is what I see in the master's monitor regarding the slave >>> >>> Non-Functioning Tablet Servers >>> The following tablet servers reported a status other than Online >>> >>> 10.240.203.36:9997 UNRESPONSIVE >>> >>> >>> >>> In the master log I see the following >>> >>> 2013-12-31 22:56:13,665 [master.Master] ERROR: unable to get >>> tablet server status 10.240.203.36:9997[1434a79d34404a2] >>> org.apache.thrift.transport.TTransportException: >>> java.net.NoRouteToHostException: No route to host >>> 2013-12-31 22:56:13,712 [master.Master] ERROR: unable to get >>> tablet server status 10.240.203.36:9997[1434a79d34404a2] >>> org.apache.thrift.transport.TTransportException: >>> java.net.NoRouteToHostException: No route to host >>> 2013-12-31 22:56:13,802 [balancer.TableLoadBalancer] INFO : Loaded >>> class >>> org.apache.accumulo.server.master.balancer.DefaultLoadBalancer for >>> table !0 >>> 2013-12-31 22:56:13,803 [master.Master] INFO : Assigning 1 tablets >>> 2013-12-31 22:56:13,812 [master.Master] ERROR: Error processing >>> table state for store Root Tablet >>> org.apache.thrift.transport.TTransportException: >>> java.net.NoRouteToHostException: No route to host >>> at >>> org.apache.accumulo.core.client.impl.ThriftTransportPool. >>> createNewTransport(ThriftTransportPool.java:475) >>> at >>> org.apache.accumulo.core.client.impl.ThriftTransportPool. >>> getTransport(ThriftTransportPool.java:464) >>> at >>> org.apache.accumulo.core.client.impl.ThriftTransportPool. >>> getTransport(ThriftTransportPool.java:441) >>> at >>> org.apache.accumulo.core.client.impl.ThriftTransportPool. >>> getTransportWithDefaultTimeout(ThriftTransportPool.java:366) >>> >>> >>> >>> In the slave's tserver.log all I see is >>> >>> 2013-12-31 22:56:34,731 [tabletserver.TabletServer] FATAL: Lost >>> tablet server lock (reason = LOCK_DELETED), exiting. >>> >>> >> -- >> >> Kurt Christensen >> P.O. Box 811 >> Westminster, MD 21158-0811 >> >> ------------------------------------------------------------------------ >> If you can't explain it simply, you don't understand it well enough. -- >> Albert Einstein >> > > --001a11c2772a57ea0604eee2dbef Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
I am probably missing something really basic so I posted b= oth the master and the slave log files:


Thanks again to everyone for the help!


On Tue, Dec = 31, 2013 at 10:20 PM, Arshak Navruzyan <arshakn@gmail.com> w= rote:
disabled selinux (iptables = already off) on both master and slave but didn't make a difference unfo= rtunately.



On Tue, Dec 31, 2013 at 9:25 PM,= Kurt Christensen <hoodel@hoodel.com> wrote:

SELINUX disabled? IPTABLES configured? I have nothing else.

Kurt

------


On 12/31/13 6:02 PM, Arshak Navruzyan wrote:
I configured a new instance with a master and a slave tserver. =A0When I do= start-all on the master, the slave doesn't come up. =A0I am wondering = if it's because I left the instance secret as the default. (I get an ex= ception when I try to change that).

This is what I see in the master's monitor regarding the slave

=A0 =A0 Non-Functioning Tablet Servers
=A0 =A0 The following tablet servers reported a status other than Online
10.240.203.36= :9997 <http:= //10.240.203.36:9997> =A0UNRESPONSIVE



In the master log I see the following

=A0 =A0 2013-12-31 22:56:13,665 [master.Master] ERROR: unable to get
=A0 =A0 tablet server status 10.240.203.36:9997[1434a79d34404a2]
=A0 =A0 org.apache.thrift.transport.TTransportException:
=A0 =A0 java.net.N= oRouteToHostException: No route to host
=A0 =A0 2013-12-31 22:56:13,712 [master.Master] ERROR: unable to get
=A0 =A0 tablet server status 10.240.203.36:9997[1434a79d34404a2]
=A0 =A0 org.apache.thrift.transport.TTransportException:
=A0 =A0 java.net.N= oRouteToHostException: No route to host
=A0 =A0 2013-12-31 22:56:13,802 [balancer.TableLoadBalancer] INFO : Loaded<= br> =A0 =A0 class
=A0 =A0 org.apache.accumulo.server.master.balancer.DefaultLoa= dBalancer for
=A0 =A0 table !0
=A0 =A0 2013-12-31 22:56:13,803 [master.Master] INFO : Assigning 1 tablets<= br> =A0 =A0 2013-12-31 22:56:13,812 [master.Master] ERROR: Error processing
=A0 =A0 table state for store Root Tablet
=A0 =A0 org.apache.thrift.transport.TTransportException:
=A0 =A0 java.net.N= oRouteToHostException: No route to host
=A0 =A0 =A0 =A0 =A0 =A0 at
=A0 =A0 org.apache.accumulo.core.client.impl.ThriftTransportP= ool.createNewTransport(ThriftTransportPool.java:475)
=A0 =A0 =A0 =A0 =A0 =A0 at
=A0 =A0 org.apache.accumulo.core.client.impl.ThriftTransportP= ool.getTransport(ThriftTransportPool.java:464)
=A0 =A0 =A0 =A0 =A0 =A0 at
=A0 =A0 org.apache.accumulo.core.client.impl.ThriftTransportP= ool.getTransport(ThriftTransportPool.java:441)
=A0 =A0 =A0 =A0 =A0 =A0 at
=A0 =A0 org.apache.accumulo.core.client.impl.ThriftTransportP= ool.getTransportWithDefaultTimeout(ThriftTransportPool.java:3= 66)



In the slave's tserver.log all I see is

=A0 =A0 2013-12-31 22:56:34,731 [tabletserver.TabletServer] FATAL: Lost
=A0 =A0 tablet server lock (reason =3D LOCK_DELETED), exiting.


--

Kurt Christensen
P.O. Box 811
Westminster, MD 21158-0811

-------------------------------------------------------------= -----------
If you can't explain it simply, you don't understand it well enough= . -- Albert Einstein


--001a11c2772a57ea0604eee2dbef--