Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id ABBA2107E0 for ; Tue, 31 Dec 2013 23:22:36 +0000 (UTC) Received: (qmail 68328 invoked by uid 500); 31 Dec 2013 23:22:36 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 68292 invoked by uid 500); 31 Dec 2013 23:22:36 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 68283 invoked by uid 99); 31 Dec 2013 23:22:36 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 31 Dec 2013 23:22:36 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy includes SPF record at spf.trusted-forwarder.org) Received: from [74.125.83.49] (HELO mail-ee0-f49.google.com) (74.125.83.49) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 31 Dec 2013 23:22:32 +0000 Received: by mail-ee0-f49.google.com with SMTP id c41so5674425eek.22 for ; Tue, 31 Dec 2013 15:22:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=yFDR97vXfbZ59uLMc7tZXKemGgxMPcfeIDayWGNQLo4=; b=lkOWCZ5uCBUSOS95P/2X4XdY3NaRL8fUPF1Di4PDYgfr0Xpo3l2LmAtreqRl5bMMAP jIkxBxdKMg5a6EDCHAvDGBEn0fHipmuoYQhSVM7sT9Mr184gQ0XejSK939OpaT0QkTD4 cdiqL1wL3Cal7v3295ox0aC61PvBE3SPgrDMOdKGvn0oVHd5HL4MGxzLasn0YBEi8Do7 Xyprg8CYlDU190gR3Dh7fN91OSUkK27CzpU71N9mFkMcKEXYtfTMRxC/ls/MZbdo494D kCGGhwqyU9JRA4PnYQi37YBcP2AqpSie3mqb25M7QTvQdzcYIU3rFX9eSYRt2a58QLSl QqXQ== X-Gm-Message-State: ALoCoQnkRe86CAKox3YeRWkEfqkmw22Zt7rV6TSPtb+u7Pz6IFILeE0G4aona/3gbPqDcJST154X MIME-Version: 1.0 X-Received: by 10.15.74.200 with SMTP id j48mr8525562eey.102.1388532131262; Tue, 31 Dec 2013 15:22:11 -0800 (PST) Received: by 10.14.105.195 with HTTP; Tue, 31 Dec 2013 15:22:11 -0800 (PST) Received: by 10.14.105.195 with HTTP; Tue, 31 Dec 2013 15:22:11 -0800 (PST) In-Reply-To: References: Date: Tue, 31 Dec 2013 17:22:11 -0600 Message-ID: Subject: Re: slave tserver not responding From: Sean Busbey To: Accumulo User List Content-Type: multipart/alternative; boundary=001a11c37d3871840904eedcd437 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c37d3871840904eedcd437 Content-Type: text/plain; charset=UTF-8 Oh! What does "hadoop version" report? On Dec 31, 2013 5:17 PM, "Arshak Navruzyan" wrote: > IPv4. I am using ip address when I ssh. masters and slaves files in the > conf directory also have ip addresses. > > > On Tue, Dec 31, 2013 at 3:13 PM, Sean Busbey wrote: > >> when you ssh, are you using hostnames or hte ip addresses? >> >> is IPv6 present? >> >> >> On Tue, Dec 31, 2013 at 5:11 PM, Arshak Navruzyan wrote: >> >>> Accumulo 1.5. Nothing in the *.err or *.out files on either the master >>> or the slave. >>> >>> Needless to say I can ssh from the master to the slave. >>> >>> Thanks! >>> >>> >>> On Tue, Dec 31, 2013 at 3:05 PM, Christopher wrote: >>> >>>> What version? >>>> Also, check the contents of the *.err and *.out logs. >>>> >>>> >>>> -- >>>> Christopher L Tubbs II >>>> http://gravatar.com/ctubbsii >>>> >>>> >>>> On Tue, Dec 31, 2013 at 6:02 PM, Arshak Navruzyan wrote: >>>> >>>>> I configured a new instance with a master and a slave tserver. When I >>>>> do start-all on the master, the slave doesn't come up. I am wondering if >>>>> it's because I left the instance secret as the default. (I get an exception >>>>> when I try to change that). >>>>> >>>>> This is what I see in the master's monitor regarding the slave >>>>> >>>>> Non-Functioning Tablet Servers >>>>> The following tablet servers reported a status other than Online >>>>> >>>>> 10.240.203.36:9997 UNRESPONSIVE >>>>> In the master log I see the following >>>>> >>>>> 2013-12-31 22:56:13,665 [master.Master] ERROR: unable to get tablet >>>>> server status 10.240.203.36:9997[1434a79d34404a2] >>>>> org.apache.thrift.transport.TTransportException: >>>>> java.net.NoRouteToHostException: No route to host >>>>> 2013-12-31 22:56:13,712 [master.Master] ERROR: unable to get tablet >>>>> server status 10.240.203.36:9997[1434a79d34404a2] >>>>> org.apache.thrift.transport.TTransportException: >>>>> java.net.NoRouteToHostException: No route to host >>>>> 2013-12-31 22:56:13,802 [balancer.TableLoadBalancer] INFO : Loaded >>>>> class org.apache.accumulo.server.master.balancer.DefaultLoadBalancer for >>>>> table !0 >>>>> 2013-12-31 22:56:13,803 [master.Master] INFO : Assigning 1 tablets >>>>> 2013-12-31 22:56:13,812 [master.Master] ERROR: Error processing table >>>>> state for store Root Tablet >>>>> org.apache.thrift.transport.TTransportException: >>>>> java.net.NoRouteToHostException: No route to host >>>>> at >>>>> org.apache.accumulo.core.client.impl.ThriftTransportPool.createNewTransport(ThriftTransportPool.java:475) >>>>> at >>>>> org.apache.accumulo.core.client.impl.ThriftTransportPool.getTransport(ThriftTransportPool.java:464) >>>>> at >>>>> org.apache.accumulo.core.client.impl.ThriftTransportPool.getTransport(ThriftTransportPool.java:441) >>>>> at >>>>> org.apache.accumulo.core.client.impl.ThriftTransportPool.getTransportWithDefaultTimeout(ThriftTransportPool.java:366) >>>>> >>>>> >>>>> >>>>> In the slave's tserver.log all I see is >>>>> >>>>> 2013-12-31 22:56:34,731 [tabletserver.TabletServer] FATAL: Lost tablet >>>>> server lock (reason = LOCK_DELETED), exiting. >>>>> >>>>> >>>>> >>>> >>>> >>> >> >> >> -- >> Sean >> > > --001a11c37d3871840904eedcd437 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


Oh! What does "hadoop version" report?

On Dec 31, 2013 5:17 PM, "Arshak Navruzyan&= quot; <arshakn@gmail.com> wr= ote:
IPv4. =C2=A0I am using ip address when I ssh. =C2=A0master= s and slaves files in the conf directory also have ip addresses.


On Tue, Dec 31, 20= 13 at 3:13 PM, Sean Busbey <busbey+ml@clouderagovt.com> wrote:
when you ssh, are you using= hostnames or hte ip addresses?

is IPv6 present?


On = Tue, Dec 31, 2013 at 5:11 PM, Arshak Navruzyan <arshakn@gmail.com><= /span> wrote:
Accumulo 1.5. =C2=A0Nothing= in the *.err or *.out files on either the master or the slave.

Needless to say I can ssh from the master to the slave.

Thanks!=C2=A0


On Tue, Dec 31, 2013 at 3:05 PM, Christo= pher <ctubbsii@apache.org> wrote:
What version?
Also, check the contents of the *.err and= *.out logs.


--<= br>Christopher L Tubbs II
http://gravatar.com/ctubbsii


On Tue, Dec 31, 2013 at 6:02 PM, Arshak = Navruzyan <arshakn@gmail.com> wrote:
I configured a new instance with a master and a slave tser= ver. =C2=A0When I do start-all on the master, the slave doesn't come up= . =C2=A0I am wondering if it's because I left the instance secret as th= e default. (I get an exception when I try to change that). =C2=A0

This is what I see in the master's monitor regarding the= slave
Non-Functioning= =C2=A0Tablet=C2=A0Servers
The following tablet servers re= ported a status other than Online
10.240.203.36:9997 UNRESPONSIVE

In the master log I se= e the following

2013-12-31 22:56:13,665 [master.Master] = ERROR: unable to get tablet server status 10.240.203.36:9997[1434a79d34404a= 2] org.apache.thrift.transport.TTransportException: java.net.NoRouteToHostE= xception: No route to host
2013-12-31 22:56:13,712 [master.Master] ERROR: unable to ge= t tablet server status 10.240.203.36:9997[1434a79d34404a2] org.apache.thrif= t.transport.TTransportException: java.net.NoRouteToHostException: No route = to host
2013-12-31 22:56:13,802 [balancer.TableLoadBalancer] INFO := Loaded class org.apache.accumulo.server.master.balancer.DefaultLoadBalance= r for table !0
2013-12-31 22:56:13,803 [master.Master]= INFO : Assigning 1 tablets
2013-12-31 22:56:13,812 [master.Master] ERROR: Error proces= sing table state for store Root Tablet
org.apache.thri= ft.transport.TTransportException: java.net.NoRouteToHostException: No route= to host
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.accumulo.core.cli= ent.impl.ThriftTransportPool.createNewTransport(ThriftTransportPool.java:47= 5)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.accumulo.= core.client.impl.ThriftTransportPool.getTransport(ThriftTransportPool.java:= 464)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.accumulo.core.cli= ent.impl.ThriftTransportPool.getTransport(ThriftTransportPool.java:441)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.accumulo.core.c= lient.impl.ThriftTransportPool.getTransportWithDefaultTimeout(ThriftTranspo= rtPool.java:366)


In the slave's ts= erver.log all I see is

2013-12-31 22:56:34,731 [tabletse= rver.TabletServer] FATAL: Lost tablet server lock (reason =3D LOCK_DELETED)= , exiting.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0





<= /div>--
Sean
<= /div>

--001a11c37d3871840904eedcd437--