accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ott, Charles H." <CHARLES.H....@saic.com>
Subject Thread "shell" Stuck on IO
Date Wed, 17 Oct 2012 20:58:29 GMT
I am using a VMware ESXi 4.1 server  with Cloudbase(Accumulo)  on RHEL5.

I cannot start with a fresh install because I am somewhat required to
use the preconfigured image on the vm. (business rules out of my hands) 

Unfortunately the support for this preconfigured instance is not
available and I am tasked with getting it working anyway... 

 

I am able to log into the shell and view the tables, however if  I
attempt to create a table or perform a scan, a line return is shown and
then it just hangs there until finally throwing the following error:



WARN thread "shell" stuck on IO to ssdev:9999:9999 (0) for at least
120044 ms.

 

I did also discover that 9999 is the property: master.port.client in my
conf/accumulo-site.xml

 

There is also an event log that was added to the VM with web based UI
reporting:



Unable to recover
192.168.0.130:11224/b4da830b-8ecb-4868-a480-35a39f4af17a(java.io.IOExcep
tion: org.apache.thrift.transport.TTransportException:
java.net.ConnectException: Connection timed out)

         java.io.IOException:
org.apache.thrift.transport.TTransportException:
java.net.ConnectException: Connection timed out

                 at
cloudbase.server.tabletserver.log.RemoteLogger.<init>(RemoteLogger.java:
75)

                 at
cloudbase.server.master.CoordinateRecoveryTask$RecoveryJob.startCopy(Coo
rdinateRecoveryTask.java:109)

                 at
cloudbase.server.master.CoordinateRecoveryTask$RecoveryJob.access$400(Co
ordinateRecoveryTask.java:93)

                 at
cloudbase.server.master.CoordinateRecoveryTask.recover(CoordinateRecover
yTask.java:279)

                 at
cloudbase.server.master.Master$TabletGroupWatcher.run(Master.java:1155)

         Caused by: org.apache.thrift.transport.TTransportException:
java.net.ConnectException: Connection timed out

                 at
cloudbase.core.client.impl.ThriftTransportPool.createNewTransport(Thrift
TransportPool.java:428)

                 at
cloudbase.core.client.impl.ThriftTransportPool.getTransport(ThriftTransp
ortPool.java:415)

                 at
cloudbase.core.client.impl.ThriftTransportPool.getTransport(ThriftTransp
ortPool.java:392)

                 at
cloudbase.core.util.ThriftUtil.getClient(ThriftUtil.java:58)

                 at
cloudbase.server.tabletserver.log.RemoteLogger.<init>(RemoteLogger.java:
73)

                 ... 4 more

         Caused by: java.net.ConnectException: Connection timed out

                 at sun.nio.ch.Net.connect(Native Method)

                 at
sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:500)

                 at
sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:81)

                 at
sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:65)

                 at
cloudbase.core.util.TTimeoutTransport.create(TTimeoutTransport.java:23)

                 at
cloudbase.core.client.impl.ThriftTransportPool.createNewTransport(Thrift
TransportPool.java:426)

                 ... 8 more

 

 

I have seen posts relating this to the walogs folder not being
available, but I have checked that and the .lock file is being created
automatically.  

A #netstat | grep 9999 shows no processes using 9999 before logging into
the shell... so Im not sure there is a port conflict either.

 

Any thoughts on the matter would be greatly appreciated. 


Mime
View raw message