accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Accumulo and CDH4 -- "no tablet servers"
Date Tue, 09 Apr 2013 00:05:06 GMT
No worries. Glad we could point you in the right direction.

Make sure you don't overlook the exception trying to load the native 
map. This should be as simple as running `make` in 
'server/src/main/c++'. That will place the native map shared object in 
'lib/native/map'.

Hopefully netstat will get you the rest of the way through your problem. 
Let us know!

- Josh

On 04/08/2013 07:08 PM, Newman, Elise wrote:
> One of my tserver__.log files contains the following (the others look similar):
>
> 4/8/13 @ 3:26 - got the following in tserver_apoc.debug.log:
> 2013-04-08 15:18:07,398 [server.Accumulo] INFO : tserver starting
> 2013-04-08 15:18:07,513 [client.ZooKeeperInstance] DEBUG: Trying to read instance id
from /accumulo/instance_id
> 2013-04-08 15:18:07,513 [server.Accumulo] INFO : Instance e31f5326-875c-469e-94cd-bf3a2951ba71
> 2013-04-08 15:18:07,515 [server.Accumulo] INFO : Data Version 5
> 2013-04-08 15:18:07,516 [server.Accumulo] INFO : Attempting to talk to zookeeper
> 2013-04-08 15:18:07,518 [zookeeper.ZooSession] DEBUG: Connecting to trinity:2181 with
timeout 30000 with auth
> 2013-04-08 15:18:07,662 [server.Accumulo] INFO : Zookeeper connected and initialized,
attemping to talk to HDFS
> 2013-04-08 15:18:07,688 [server.Accumulo] INFO : Connected to HDFS
> 2013-04-08 15:18:07,698 [server.Accumulo] INFO : gc.cycle.delay = 5m
> 2013-04-08 15:18:07,699 [server.Accumulo] INFO : gc.cycle.start = 30s
> 2013-04-08 15:18:07,699 [server.Accumulo] INFO : gc.port.client = 50091
> 2013-04-08 15:18:07,699 [server.Accumulo] INFO : gc.threads.delete = 16
> 2013-04-08 15:18:07,700 [server.Accumulo] INFO : general.classpaths =
>    	$ACCUMULO_HOME/server/target/classes/,
>    	$ACCUMULO_HOME/core/target/classes/,
>    	$ACCUMULO_HOME/start/target/classes/,
>    	$ACCUMULO_HOME/fate/target/classes/,
>    	$ACCUMULO_HOME/proxy/target/classes/,
>    	$ACCUMULO_HOME/examples/target/classes/,
>    	$ACCUMULO_HOME/lib/[^.].$ACCUMULO_VERSION.jar,
>    	$ACCUMULO_HOME/lib/[^.].*.jar,
>    	$ZOOKEEPER_HOME/zookeeper[^.].*.jar,
>    	$HADOOP_CONF_DIR,
>    	$HADOOP_PREFIX/[^.].*.jar,
>    	$HADOOP_PREFIX/lib/[^.].*.jar,
>    	$HADOOP_HDFS_HOME/.*.jar,
>    	$HADOOP_HDFS_HOME/lib/.*.jar,
>    	$HADOOP_MAPREDUCE_HOME/.*.jar,
>    	$HADOOP_MAPREDUCE_HOME/lib/.*.jar,
>    	$HADOOP_YARN_HOME/.*.jar,
>    	$HADOOP_YARN_HOME/lib/.*.jar,
>
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : general.dynamic.classpaths = $ACCUMULO_HOME/lib/ext/[^.].*.jar
>
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : general.kerberos.keytab =
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : general.kerberos.principal =
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : general.rpc.timeout = 120s
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : general.vfs.cache.dir = /tmp/accumulo-vfs-cache-root
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : general.vfs.classpaths =
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : instance.dfs.dir = /accumulo
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : instance.dfs.uri =
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : instance.secret = <hidden>
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : instance.security.authenticator = org.apache.accumulo.server.security.handler.ZKAuthenticator
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : instance.security.authorizor = org.apache.accumulo.server.security.handler.ZKAuthorizor
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : instance.security.permissionHandler
= org.apache.accumulo.server.security.handler.ZKPermHandler
> 2013-04-08 15:18:07,701 [server.Accumulo] INFO : instance.zookeeper.host = trinity:2181
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : instance.zookeeper.timeout = 30s
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : logger.dir.walog = /var/accumulo/walogs
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : master.bulk.retries = 3
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : master.bulk.threadpool.size = 5
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : master.bulk.timeout = 5m
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : master.fate.threadpool.size = 4
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : master.port.client = 9999
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : master.recovery.delay = 10s
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : master.recovery.max.age = 60m
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : master.recovery.time.max = 30m
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : master.server.threadcheck.time = 1s
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : master.server.threads.minimum = 20
> 2013-04-08 15:18:07,702 [server.Accumulo] INFO : master.tablet.balancer = org.apache.accumulo.server.master.balancer.TableLoadBalancer
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : master.walog.closer.implementation =
org.apache.accumulo.server.master.recovery.HadoopLogCloser
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : monitor.banner.background = #304065
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : monitor.banner.color = #c4c4c4
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : monitor.banner.text =
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : monitor.port.client = 50095
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : monitor.port.log4j = 4560
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : monitor.ssl.keyStore =
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : monitor.ssl.keyStorePassword = <hidden>
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : monitor.ssl.trustStore =
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : monitor.ssl.trustStorePassword = <hidden>
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : table.balancer = org.apache.accumulo.server.master.balancer.DefaultLoadBalancer
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : table.bloom.enabled = false
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : table.bloom.error.rate = 0.5%
> 2013-04-08 15:18:07,703 [server.Accumulo] INFO : table.bloom.hash.type = murmur
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.bloom.key.functor = org.apache.accumulo.core.file.keyfunctor.RowFunctor
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.bloom.load.threshold = 1
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.bloom.size = 1048576
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.cache.block.enable = false
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.cache.index.enable = true
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.classpath.context =
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.compaction.major.everything.idle
= 1h
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.compaction.major.ratio = 3
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.compaction.minor.idle = 5m
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.compaction.minor.logs.threshold
= 3
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.failures.ignore = false
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.file.blocksize = 0B
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.file.compress.blocksize = 100K
> 2013-04-08 15:18:07,704 [server.Accumulo] INFO : table.file.compress.blocksize.index
= 128K
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : table.file.compress.type = gz
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : table.file.max = 15
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : table.file.replication = 0
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : table.file.type = rf
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : table.formatter = org.apache.accumulo.core.util.format.DefaultFormatter
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : table.groups.enabled =
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : table.interepreter = org.apache.accumulo.core.util.interpret.DefaultScanInterpreter
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : table.scan.max.memory = 512K
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : table.security.scan.visibility.default
=
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : table.split.threshold = 1G
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : table.walog.enabled = true
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : trace.login.password = <hidden>
> 2013-04-08 15:18:07,705 [server.Accumulo] INFO : trace.password = <hidden>
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : trace.port.client = 12234
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : trace.principal = root
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : trace.table = trace
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : trace.user = root
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : tserver.archive.walogs = false
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : tserver.bloom.load.concurrent.max =
4
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : tserver.bulk.assign.threads = 1
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : tserver.bulk.process.threads = 1
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : tserver.bulk.retry.max = 5
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : tserver.bulk.timeout = 5m
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : tserver.cache.data.size = 7M
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : tserver.cache.index.size = 20M
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : tserver.client.timeout = 3s
> 2013-04-08 15:18:07,706 [server.Accumulo] INFO : tserver.compaction.major.concurrent.max
= 3
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.compaction.major.delay = 30s
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.compaction.major.thread.files.open.max
= 10
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.compaction.minor.concurrent.max
= 4
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.default.blocksize = 1M
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.dir.memdump = /tmp
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.files.open.idle = 1m
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.hold.time.max = 5m
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.memory.lock = false
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.memory.manager = org.apache.accumulo.server.tabletserver.LargestFirstMemoryManager
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.memory.maps.max = 80M
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.memory.maps.native.enabled =
true
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.metadata.readahead.concurrent.max
= 8
> 2013-04-08 15:18:07,707 [server.Accumulo] INFO : tserver.migrations.concurrent.max =
1
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.monitor.fs = true
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.mutation.queue.max = 256K
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.port.client = 9997
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.port.search = false
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.readahead.concurrent.max = 16
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.recovery.concurrent.max = 2
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.scan.files.open.max = 100
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.server.message.size.max = 1G
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.server.threadcheck.time = 1s
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.server.threads.minimum = 20
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.session.idle.max = 1m
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.sort.buffer.size = 50M
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.tablet.split.midpoint.files.max
= 30
> 2013-04-08 15:18:07,708 [server.Accumulo] INFO : tserver.wal.blocksize = 0
> 2013-04-08 15:18:07,709 [server.Accumulo] INFO : tserver.wal.replication = 0
> 2013-04-08 15:18:07,709 [server.Accumulo] INFO : tserver.walog.max.size = 100M
> 2013-04-08 15:18:07,709 [server.Accumulo] INFO : tserver.workq.threads = 2
> 2013-04-08 15:18:07,745 [metrics.MetricsConfiguration] DEBUG: Loading config file: /home/d/apps/accumulo-1.6.0-bleeding-edge/apache-accumulo-1.6.0-SNAPSHOT/conf/accumulo-metrics.xml
> 2013-04-08 15:18:07,783 [metrics.MetricsConfiguration] DEBUG: Metrics collection enabled=false
> 2013-04-08 15:18:07,784 [metrics.MetricsConfiguration] DEBUG: Loading config file: /home/d/apps/accumulo-1.6.0-bleeding-edge/apache-accumulo-1.6.0-SNAPSHOT/conf/accumulo-metrics.xml
> 2013-04-08 15:18:07,790 [metrics.MetricsConfiguration] DEBUG: Metrics collection enabled=false
> 2013-04-08 15:18:07,826 [tabletserver.TabletServer] INFO : Tablet server starting on
apoc
> 2013-04-08 15:18:07,841 [master.Master] INFO : Loaded class : org.apache.accumulo.server.security.handler.ZKAuthorizor
> 2013-04-08 15:18:07,844 [master.Master] INFO : Loaded class : org.apache.accumulo.server.security.handler.ZKAuthenticator
> 2013-04-08 15:18:07,846 [master.Master] INFO : Loaded class : org.apache.accumulo.server.security.handler.ZKPermHandler
> 2013-04-08 15:18:07,860 [util.FileSystemMonitor] INFO : Filesystem monitor started
> 2013-04-08 15:18:07,870 [tabletserver.TabletServer] DEBUG: gc ParNew=0.03(+0.03) secs
ConcurrentMarkSweep=0.00(+0.00) secs freemem=118,996,184(+118,996,184) totalmem=132,055,040
> 2013-04-08 15:18:07,871 [tabletserver.NativeMap] ERROR: Failed to load native map library
/home/d/apps/accumulo-1.6.0-bleeding-edge/apache-accumulo-1.6.0-SNAPSHOT/lib/native/map/libNativeMap-Linux-amd64-64.so
> java.lang.UnsatisfiedLinkError: Can't load library: /home/d/apps/accumulo-1.6.0-bleeding-edge/apache-accumulo-1.6.0-SNAPSHOT/lib/native/map/libNativeMap-Linux-amd64-64.so
>      at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1706)
>      at java.lang.Runtime.load0(Runtime.java:770)
>      at java.lang.System.load(System.java:1003)
>      at org.apache.accumulo.server.tabletserver.NativeMap.loadNativeLib(NativeMap.java:144)
>      at org.apache.accumulo.server.tabletserver.NativeMap.<clinit>(NativeMap.java:156)
>      at org.apache.accumulo.server.tabletserver.TabletServerResourceManager.<init>(TabletServerResourceManager.java:148)
>      at org.apache.accumulo.server.tabletserver.TabletServer.config(TabletServer.java:3170)
>      at org.apache.accumulo.server.tabletserver.TabletServer.main(TabletServer.java:3281)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>      at java.lang.reflect.Method.invoke(Method.java:597)
>      at org.apache.accumulo.start.Main$1.run(Main.java:101)
>      at java.lang.Thread.run(Thread.java:662)
> 2013-04-08 15:18:08,018 [tabletserver.TabletServerResourceManager] DEBUG: Loaded memory
manager : org.apache.accumulo.server.tabletserver.LargestFirstMemoryManager
> 2013-04-08 15:18:08,035 [trace.ZooTraceClient] DEBUG: Scanning trace hosts in zookeeper:
/accumulo/e31f5326-875c-469e-94cd-bf3a2951ba71/tracers
> 2013-04-08 15:18:08,035 [trace.ZooTraceClient] DEBUG: Trace hosts: []
> 2013-04-08 15:18:08,057 [metrics.MetricsConfiguration] DEBUG: Loading config file: /home/d/apps/accumulo-1.6.0-bleeding-edge/apache-accumulo-1.6.0-SNAPSHOT/conf/accumulo-metrics.xml
> 2013-04-08 15:18:08,063 [metrics.MetricsConfiguration] DEBUG: Metrics collection enabled=false
> 2013-04-08 15:18:08,065 [metrics.MetricsConfiguration] DEBUG: Loading config file: /home/d/apps/accumulo-1.6.0-bleeding-edge/apache-accumulo-1.6.0-SNAPSHOT/conf/accumulo-metrics.xml
> 2013-04-08 15:18:08,071 [metrics.MetricsConfiguration] DEBUG: Metrics collection enabled=false
> 2013-04-08 15:18:08,072 [tabletserver.TabletServer] DEBUG: org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler
created
> 2013-04-08 15:18:08,208 [metrics.MetricsConfiguration] DEBUG: Loading config file: /home/d/apps/accumulo-1.6.0-bleeding-edge/apache-accumulo-1.6.0-SNAPSHOT/conf/accumulo-metrics.xml
> 2013-04-08 15:18:08,214 [metrics.MetricsConfiguration] DEBUG: Metrics collection enabled=false
> 2013-04-08 15:18:08,216 [tabletserver.TabletServer] INFO : port = 9997
> 2013-04-08 15:18:08,280 [tabletserver.TabletServer] DEBUG: Obtained tablet server lock
/accumulo/e31f5326-875c-469e-94cd-bf3a2951ba71/tservers/192.168.210.89:9997/zlock-0000000000
> 2013-04-08 15:18:08,713 [server.Accumulo] WARN : System swappiness setting is greater
than ten (60) which can cause time-sensitive operations to be delayed.  Accumulo is time sensitive
because it needs to maintain distributed lock agreement.
> 2013-04-08 15:18:08,865 [tabletserver.TabletServer] DEBUG: gc ParNew=0.03(+0.00) secs
ConcurrentMarkSweep=0.00(+0.00) secs freemem=121,318,224(+2,322,040) totalmem=132,055,040
> 2013-04-08 15:18:15,910 [tabletserver.TabletServer] DEBUG: Got loadTablet message from
user: !SYSTEM
> 2013-04-08 15:18:15,955 [tabletserver.TabletServer] INFO : Loading tablet !0;!0<<
> 2013-04-08 15:18:15,958 [tabletserver.TabletServer] INFO : apoc/192.168.210.89:9997:
got assignment from master: !0;!0<<
> 2013-04-08 15:18:15,958 [tabletserver.TabletServer] DEBUG: Got loadTablet message from
user: !SYSTEM
> 2013-04-08 15:18:15,960 [tabletserver.TabletServer] DEBUG: Loading extent: !0;!0<<
> 2013-04-08 15:18:15,960 [tabletserver.TabletServer] DEBUG: Got loadTablet message from
user: !SYSTEM
> 2013-04-08 15:18:15,965 [tabletserver.TabletServer] DEBUG: verifying extent !0;!0<<
> 2013-04-08 15:18:15,977 [state.ZooTabletStateStore] DEBUG: Returning root tablet state:
!0;!0<<@(192.168.210.89:9997[13ddc40d693c74c],null,null)
> 2013-04-08 15:18:15,996 [util.MetadataTable] INFO : Scanning logging entries for !0;!0<<
> 2013-04-08 15:18:15,996 [util.MetadataTable] INFO : Getting logs for root tablet from
zookeeper
> 2013-04-08 15:18:15,998 [util.MetadataTable] INFO : Returning logs [] for extent !0;!0<<
> 2013-04-08 15:18:15,998 [tabletserver.Tablet] DEBUG: got [] for logs for !0;!0<<
> 2013-04-08 15:18:16,384 [constraints.ConstraintChecker] DEBUG: Loaded constraint org.apache.accumulo.server.constraints.MetadataConstraints
for !0
> 2013-04-08 15:18:16,389 [tabletserver.Tablet] TABLET_HIST: !0;!0<< opened
> 2013-04-08 15:18:16,397 [state.ZooTabletStateStore] DEBUG: Returning root tablet state:
!0;!0<<@(192.168.210.89:9997[13ddc40d693c74c],null,null)
> 2013-04-08 15:18:16,450 [state.ZooStore] DEBUG: Wrote 192.168.210.89:9997|13ddc40d693c74c
to /accumulo/e31f5326-875c-469e-94cd-bf3a2951ba71/root_tablet/location
> 2013-04-08 15:18:16,492 [state.ZooStore] DEBUG: Wrote 192.168.210.89:9997|13ddc40d693c74c
to /accumulo/e31f5326-875c-469e-94cd-bf3a2951ba71/root_tablet/lastlocation
> 2013-04-08 15:18:16,492 [state.ZooStore] DEBUG: Removing /root_tablet/future_location
> 2013-04-08 15:18:16,533 [state.ZooTabletStateStore] DEBUG: Put down root tablet location
> 2013-04-08 15:18:16,536 [tabletserver.TabletServer] INFO : Root tablet loaded: !0;!0<<
> 2013-04-08 15:18:16,712 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.87:38352
!0 2 entries in 0.12 secs, nbTimes = [103 103 103.00 1]
> 2013-04-08 15:18:16,803 [tabletserver.TabletServer] DEBUG: MultiScanSess 192.168.210.87:38351
2 entries in 0.22 secs (lookup_time:0.11 secs tablets:1 ranges:1)
> 2013-04-08 15:18:16,864 [tabletserver.TabletServer] DEBUG: gc ParNew=0.04(+0.01) secs
ConcurrentMarkSweep=0.00(+0.00) secs freemem=114,527,520(-5,732,744) totalmem=132,055,040
> 2013-04-08 15:18:16,920 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.87:38352
!0 2 entries in 0.02 secs, nbTimes = [15 15 15.00 1]
> 2013-04-08 15:18:16,924 [log.DfsLogger] DEBUG: DfsLogger.open() begin
> 2013-04-08 15:18:16,932 [conf.Configuration] WARN : io.bytes.per.checksum is deprecated.
Instead, use dfs.bytes-per-checksum
> 2013-04-08 15:18:16,972 [crypto.CryptoModuleFactory] DEBUG: About to instantiate crypto
module NullCryptoModule
> 2013-04-08 15:18:17,040 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.87:38352
!0 2 entries in 0.00 secs, nbTimes = [4 4 4.00 1]
> 2013-04-08 15:18:17,102 [log.DfsLogger] DEBUG: Got new write-ahead log: 192.168.210.89+9997/e35cd6f9-59f5-4fad-961b-4a60b84ee67e
> 2013-04-08 15:18:17,107 [tabletserver.TabletServer] INFO : Adding 1 logs for extent !0;!0<<
as alias 1
> 2013-04-08 15:18:17,131 [tabletserver.TabletServer] DEBUG: UpSess 192.168.210.87:38351
2 in 0.271s, at=[1 1 1.00 1] ft=0.220s(pt=0.001s lt=0.217s ct=0.002s)
> 2013-04-08 15:18:17,147 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.87:38352
!0 2 entries in 0.00 secs, nbTimes = [2 2 2.00 1]
> 2013-04-08 15:18:17,169 [tabletserver.TabletServer] DEBUG: MultiScanSess 192.168.210.87:38351
2 entries in 0.01 secs (lookup_time:0.01 secs tablets:1 ranges:1)
> 2013-04-08 15:18:17,194 [tabletserver.TabletServer] DEBUG: MultiScanSess 192.168.210.87:38351
2 entries in 0.00 secs (lookup_time:0.00 secs tablets:1 ranges:1)
> 2013-04-08 15:18:17,240 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.86:53414
!0 8 entries in 0.00 secs, nbTimes = [3 3 3.00 1]
> 2013-04-08 15:18:17,257 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.87:38351
!0 2 entries in 0.00 secs, nbTimes = [2 2 2.00 1]
> 2013-04-08 15:18:17,271 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.86:53414
!0 0 entries in 0.00 secs, nbTimes = [1 1 1.00 1]
> 2013-04-08 15:18:17,277 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.86:53414
!0 0 entries in 0.00 secs, nbTimes = [1 1 1.00 1]
> 2013-04-08 15:18:17,364 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.87:38351
!0 2 entries in 0.00 secs, nbTimes = [2 2 2.00 1]
> 2013-04-08 15:18:17,453 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.86:53414
!0 6 entries in 0.00 secs, nbTimes = [2 2 2.00 1]
> 2013-04-08 15:18:17,460 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.86:53414
!0 0 entries in 0.00 secs, nbTimes = [2 2 2.00 1]
> 2013-04-08 15:18:17,465 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.86:53414
!0 0 entries in 0.00 secs, nbTimes = [1 1 1.00 1]
> 2013-04-08 15:18:17,472 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.87:38351
!0 3 entries in 0.00 secs, nbTimes = [3 3 3.00 1]
> 2013-04-08 15:18:17,584 [tabletserver.TabletServer] DEBUG: ScanSess tid 192.168.210.87:38351
!0 4 entries in 0.00 secs, nbTimes = [2 2 2.00 1]
> 2013-04-08 15:18:17,866 [tabletserver.TabletServer] DEBUG: gc ParNew=0.05(+0.01) secs
ConcurrentMarkSweep=0.00(+0.00) secs freemem=122,744,960(+8,217,440) totalmem=132,055,040
> 2013-04-08 15:19:07,897 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=0.00340271MB
(3568), Free=19.996597MB (20967952), Max=20.0MB (20971520), Counts: Blocks=3, Access=6, Hit=0,
Miss=6, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN,
Duplicate Reads=3
> 2013-04-08 15:19:07,910 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=0.0020751953MB
(2176), Free=6.997925MB (7337856), Max=7.0MB (7340032), Counts: Blocks=1, Access=13, Hit=11,
Miss=2, Evictions=0, Evicted=0, Ratios: Hit Ratio=84.61538553237915%, Miss Ratio=15.384615957736969%,
Evicted/Run=NaN, Duplicate Reads=1
> 2013-04-08 15:20:07,897 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=0.00340271MB
(3568), Free=19.996597MB (20967952), Max=20.0MB (20971520), Counts: Blocks=3, Access=6, Hit=0,
Miss=6, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN,
Duplicate Reads=3
> 2013-04-08 15:20:07,910 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=0.0020751953MB
(2176), Free=6.997925MB (7337856), Max=7.0MB (7340032), Counts: Blocks=1, Access=13, Hit=11,
Miss=2, Evictions=0, Evicted=0, Ratios: Hit Ratio=84.61538553237915%, Miss Ratio=15.384615957736969%,
Evicted/Run=NaN, Duplicate Reads=1
> 2013-04-08 15:21:07,897 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=0.00340271MB
(3568), Free=19.996597MB (20967952), Max=20.0MB (20971520), Counts: Blocks=3, Access=6, Hit=0,
Miss=6, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN,
Duplicate Reads=3
> 2013-04-08 15:21:07,910 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=0.0020751953MB
(2176), Free=6.997925MB (7337856), Max=7.0MB (7340032), Counts: Blocks=1, Access=13, Hit=11,
Miss=2, Evictions=0, Evicted=0, Ratios: Hit Ratio=84.61538553237915%, Miss Ratio=15.384615957736969%,
Evicted/Run=NaN, Duplicate Reads=1
> 2013-04-08 15:22:07,897 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=0.00340271MB
(3568), Free=19.996597MB (20967952), Max=20.0MB (20971520), Counts: Blocks=3, Access=6, Hit=0,
Miss=6, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN,
Duplicate Reads=3
> 2013-04-08 15:22:07,910 [cache.LruBlockCache] DEBUG: Cache Stats: Sizes: Total=0.0020751953MB
(2176), Free=6.997925MB (7337856), Max=7.0MB (7340032), Counts: Blocks=1, Access=13, Hit=11,
Miss=2, Evictions=0, Evicted=0, Ratios: Hit Ratio=84.61538553237915%, Miss Ratio=15.384615957736969%,
Evicted/Run=NaN, Duplicate Reads=1
>
> With respect to the sanity check mentioned below, I've built against cdh4 and have accumulo
pointing to ClouderaManager-installed instances of Hadoop and Zookeeper, whose locations I've
recorded in accumulo-env.sh:
>
> if [ -z "$HADOOP_HOME" ]
> then
>     test -z "$HADOOP_PREFIX"  	&& export HADOOP_PREFIX=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop
> else
>     HADOOP_PREFIX="$HADOOP_HOME"
>     unset HADOOP_HOME
> fi
> # hadoop-1.0:
> # test -z "$HADOOP_CONF_DIR"   	&& export HADOOP_CONF_DIR="$HADOOP_PREFIX/conf"
> # hadoop-2.0:
> test -z "$HADOOP_CONF_DIR" 	&& export HADOOP_CONF_DIR="$HADOOP_PREFIX/etc/hadoop"
> # cdh4
> export HADOOP_HDFS_HOME=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop-hdfs
> export HADOOP_MAPREDUCE_HOME=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop-mapreduce
> export HADOOP_YARN_HOME=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/hadoop-yarn
>
> test -z "$JAVA_HOME"         	&& export JAVA_HOME=/usr/java/jdk1.6.0_31
> test -z "$ZOOKEEPER_HOME"    	&& export ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/lib/zookeeper
> test -z "$ACCUMULO_LOG_DIR"  	&& export ACCUMULO_LOG_DIR=$ACCUMULO_HOME/logs
> if [ -f ${ACCUMULO_HOME}/conf/accumulo.policy ]
> then
>     POLICY="-Djava.security.manager -Djava.security.policy=${ACCUMULO_HOME}/conf/accumulo.policy"
> fi
> test -z "$ACCUMULO_TSERVER_OPTS" && export ACCUMULO_TSERVER_OPTS="${POLICY} -Xmx128m
-Xms128m "
> test -z "$ACCUMULO_MASTER_OPTS"  && export ACCUMULO_MASTER_OPTS="${POLICY} -Xmx128m
-Xms128m"
> test -z "$ACCUMULO_MONITOR_OPTS" && export ACCUMULO_MONITOR_OPTS="${POLICY} -Xmx64m
-Xms64m"
> test -z "$ACCUMULO_GC_OPTS"  	&& export ACCUMULO_GC_OPTS="-Xmx64m -Xms64m"
> test -z "$ACCUMULO_GENERAL_OPTS" && export ACCUMULO_GENERAL_OPTS="-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75"
> test -z "$ACCUMULO_OTHER_OPTS"   && export ACCUMULO_OTHER_OPTS="-Xmx128m -Xms64m"
> export ACCUMULO_LOG_HOST=`(grep -v '^#' $ACCUMULO_HOME/conf/monitor ; echo localhost
) 2>/dev/null | head -1`
>
> I then add them to general.classpaths in accumulo-site.xml:
> <property>
>      <name>general.classpaths</name>
>          <!--
>                  Add the following for hadoop-2.0
>     	$HADOOP_PREFIX/share/hadoop/common/.*.jar,
>     	$HADOOP_PREFIX/share/hadoop/common/lib/.*.jar,
>     	$HADOOP_PREFIX/share/hadoop/hdfs/.*.jar,
>     	$HADOOP_PREFIX/share/hadoop/mapreduce/.*.jar,
>     	$HADOOP_PREFIX/share/hadoop/yarn/.*.jar,
>          -->
>          <value>
>    	$ACCUMULO_HOME/server/target/classes/,
>    	$ACCUMULO_HOME/core/target/classes/,
>    	$ACCUMULO_HOME/start/target/classes/,
>    	$ACCUMULO_HOME/fate/target/classes/,
>    	$ACCUMULO_HOME/proxy/target/classes/,
>    	$ACCUMULO_HOME/examples/target/classes/,
>    	$ACCUMULO_HOME/lib/[^.].$ACCUMULO_VERSION.jar,
>    	$ACCUMULO_HOME/lib/[^.].*.jar,
>    	$ZOOKEEPER_HOME/zookeeper[^.].*.jar,
>    	$HADOOP_CONF_DIR,
>    	$HADOOP_PREFIX/[^.].*.jar,
>    	$HADOOP_PREFIX/lib/[^.].*.jar,
>    	$HADOOP_HDFS_HOME/.*.jar,
>    	$HADOOP_HDFS_HOME/lib/.*.jar,
>    	$HADOOP_MAPREDUCE_HOME/.*.jar,
>    	$HADOOP_MAPREDUCE_HOME/lib/.*.jar,
>    	$HADOOP_YARN_HOME/.*.jar,
>    	$HADOOP_YARN_HOME/lib/.*.jar,
>          </value>
> </property>
>
> I've verified that these jars are on the classpath (with accumulo classpath), but am
not sure if I have any I don't need.
>
> I looked at http://localhost:50095/log and found the following 3 errors, in this order:
>
> org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address
0.0.0.0/0.0.0.0:50091.
> 	org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address
0.0.0.0/0.0.0.0:50091.
> 		at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:89)
> 		at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:68)
> 		at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:61)
> 		at org.apache.accumulo.server.util.TServerUtils.startHsHaServer(TServerUtils.java:215)
> 		at org.apache.accumulo.server.util.TServerUtils.startTServer(TServerUtils.java:277)
> 		at org.apache.accumulo.server.gc.SimpleGarbageCollector.startStatsService(SimpleGarbageCollector.java:411)
> 		at org.apache.accumulo.server.gc.SimpleGarbageCollector.run(SimpleGarbageCollector.java:210)
> 		at org.apache.accumulo.server.gc.SimpleGarbageCollector.main(SimpleGarbageCollector.java:162)
> 		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 		at java.lang.reflect.Method.invoke(Method.java:597)
> 		at org.apache.accumulo.start.Main$1.run(Main.java:101)
> 		at java.lang.Thread.run(Thread.java:662)
>
> java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Could not
create ServerSocket on address 0.0.0.0/0.0.0.0:50091.
> 	java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Could not
create ServerSocket on address 0.0.0.0/0.0.0.0:50091.
> 		at org.apache.accumulo.server.gc.SimpleGarbageCollector.startStatsService(SimpleGarbageCollector.java:414)
> 		at org.apache.accumulo.server.gc.SimpleGarbageCollector.run(SimpleGarbageCollector.java:210)
> 		at org.apache.accumulo.server.gc.SimpleGarbageCollector.main(SimpleGarbageCollector.java:162)
> 		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 		at java.lang.reflect.Method.invoke(Method.java:597)
> 		at org.apache.accumulo.start.Main$1.run(Main.java:101)
> 		at java.lang.Thread.run(Thread.java:662)
> 	Caused by: org.apache.thrift.transport.TTransportException: Could not create ServerSocket
on address 0.0.0.0/0.0.0.0:50091.
> 		at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:89)
> 		at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:68)
> 		at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:61)
> 		at org.apache.accumulo.server.util.TServerUtils.startHsHaServer(TServerUtils.java:215)
> 		at org.apache.accumulo.server.util.TServerUtils.startTServer(TServerUtils.java:277)
> 		at org.apache.accumulo.server.gc.SimpleGarbageCollector.startStatsService(SimpleGarbageCollector.java:411)
> 		... 8 more
>
> Unexpected exception, exiting
> 	java.net.UnknownHostException: Unable to find a listen port
> 		at org.apache.accumulo.server.util.TServerUtils.startServer(TServerUtils.java:126)
> 		at org.apache.accumulo.server.master.Master.run(Master.java:2107)
> 		at org.apache.accumulo.server.master.Master.main(Master.java:2230)
> 		at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 		at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 		at java.lang.reflect.Method.invoke(Method.java:597)
> 		at org.apache.accumulo.start.Main$1.run(Main.java:101)
> 		at java.lang.Thread.run(Thread.java:662)
>
> This is probably a major oversight on my part -- I apologize in advance and appreciate
any help.
>
> Thanks!
> Elise
>
>
> -----Original Message-----
> From: Josh Elser [mailto:josh.elser@gmail.com]
> Sent: Friday, April 05, 2013 3:50 PM
> To: user@accumulo.apache.org
> Subject: Re: Accumulo and CDH4 -- "no tablet servers"
>
> Elise,
>
> Check the tserver_*.{log,out} files. The exception is in the tracer log is probably a
separate issue but won't keep you from running Accumulo.
>
> A good sanity check is to make sure you don't have stray jars in lib and that ACCUMULO_HOME
is correctly set.
>
> On 04/05/2013 06:46 PM, Newman, Elise wrote:
>>
>> I am trying to properly configure Accumulo on a cluster running CDH4.
>> I can't seem to get any tablet servers up and running.
>>
>> I've checked out trunk, built it against cdh4, and ran ./accumulo init.
>>
>> Then I ran the start-all script, which produced the following output:
>>
>> ./start-all.sh
>>
>> Starting monitor on localhost
>>
>> Starting tablet servers ..... done
>>
>> Starting tablet server on *********
>>
>> Starting tablet server on *********
>>
>> 2013-04-05 13:04:05,442 [server.Accumulo] INFO : Attempting to talk to
>> zookeeper
>>
>> 2013-04-05 13:04:05,623 [server.Accumulo] INFO : Zookeeper connected
>> and initialized, attemping to talk to HDFS
>>
>> 2013-04-05 13:04:05,766 [server.Accumulo] INFO : Connected to HDFS
>>
>> Starting master on ********
>>
>> Starting garbage collector on localhost
>>
>> Starting tracer on localhost
>>
>> When I checked http://localhost:50095/tservers, however, it showed
>> that there were no tablet servers online. I looked at
>> http://localhost:50095/log, and saw the following message:
>>
>> WARN: There are no tablet servers: check that zookeeper and accumulo
>> are running.
>>
>> I get the same message when I attempt to start the accumulo shell.
>>
>> In my tracer_master.debug.log I see the following recurring exception:
>>
>> 2013-04-05 15:15:59,742 [impl.ServerClient] DEBUG: ClientService
>> request failed null, retrying ...
>>
>> org.apache.thrift.transport.TTransportException: Failed to connect to
>> a server
>>
>> at
>> org.apache.accumulo.core.client.impl.ThriftTransportPool.getAnyTranspo
>> rt(ThriftTransportPool.java:437)
>>
>> at
>> org.apache.accumulo.core.client.impl.ServerClient.getConnection(Server
>> Client.java:152)
>>
>> at
>> org.apache.accumulo.core.client.impl.ServerClient.getConnection(Server
>> Client.java:128)
>>
>> at
>> org.apache.accumulo.core.client.impl.ServerClient.getConnection(Server
>> Client.java:123)
>>
>> at
>> org.apache.accumulo.core.client.impl.ServerClient.executeRaw(ServerCli
>> ent.java:86)
>>
>> at
>> org.apache.accumulo.core.client.impl.ServerClient.execute(ServerClient
>> .java:59)
>>
>> at
>> org.apache.accumulo.server.client.HdfsZooInstance.getAuthenticatorClas
>> sName(HdfsZooInstance.java:209)
>>
>> at
>> org.apache.accumulo.server.client.HdfsZooInstance.getAuthenticator(Hdf
>> sZooInstance.java:219)
>>
>> at
>> org.apache.accumulo.server.trace.TraceServer.<init>(TraceServer.java:1
>> 78)
>>
>> at
>> org.apache.accumulo.server.trace.TraceServer.main(TraceServer.java:256
>> )
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
>> ava:39)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess
>> orImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:597)
>>
>> at org.apache.accumulo.start.Main$1.run(Main.java:101)
>>
>> at java.lang.Thread.run(Thread.java:662)
>>
>> Anyone know what could be going on?
>>
>> Thanks in advance,
>>
>> Elise
>>
>

Mime
View raw message