cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andy Cobley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5768) If a Seed can't be contacted, a new node comes up as a cluster of 1
Date Thu, 18 Jul 2013 18:34:52 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13712619#comment-13712619
] 

Andy Cobley commented on CASSANDRA-5768:
----------------------------------------

Brandon,
For info:

With the code from 5678.txt, the code above and the attached yaml file and no other nodes
in the cluster I'm getting the following trace:


sudo ./cassandra
xss =  -ea -javaagent:./../lib/jamm-0.2.5.jar -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42
-Xms1024M -Xmx1024M -Xmn256M -XX:+HeapDumpOnOutOfMemoryError
LifeintheAirAge:bin Administrator$ objc[2779]: Class JavaLaunchHelper is implemented in both
/Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/jre/lib/libinstrument.dylib.
One of the two will be used. Which one is undefined.
 INFO 10:50:38,698 Logging initialized
 INFO 10:50:38,719 JVM vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.8.0-ea
 INFO 10:50:38,719 Heap size: 1046937600/1046937600
 INFO 10:50:38,720 Classpath: ./../conf:./../build/classes/main:./../build/classes/thrift:./../lib/antlr-3.2.jar:./../lib/commons-cli-1.1.jar:./../lib/commons-codec-1.2.jar:./../lib/commons-lang-2.6.jar:./../lib/compress-lzf-0.8.4.jar:./../lib/concurrentlinkedhashmap-lru-1.3.jar:./../lib/disruptor-3.0.1.jar:./../lib/guava-13.0.1.jar:./../lib/high-scale-lib-1.1.2.jar:./../lib/jackson-core-asl-1.9.2.jar:./../lib/jackson-mapper-asl-1.9.2.jar:./../lib/jamm-0.2.5.jar:./../lib/jbcrypt-0.3m.jar:./../lib/jline-1.0.jar:./../lib/json-simple-1.1.jar:./../lib/libthrift-0.9.0.jar:./../lib/log4j-1.2.16.jar:./../lib/lz4-1.1.0.jar:./../lib/metrics-core-2.0.3.jar:./../lib/netty-3.5.9.Final.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/slf4j-api-1.7.2.jar:./../lib/slf4j-log4j12-1.7.2.jar:./../lib/snakeyaml-1.11.jar:./../lib/snappy-java-1.0.5.jar:./../lib/snaptree-0.1.jar:./../lib/thrift-server-0.2.jar:./../lib/jamm-0.2.5.jar
 INFO 10:50:38,721 JNA not found. Native methods will be disabled.
 INFO 10:50:38,739 Loading settings from file:/Users/Administrator/Documents/raspberry/cassandra2/test/cassandra-trunk/conf/cassandra.yaml
 INFO 10:50:39,159 Data files directories: [/var/lib/cassandra/data]
 INFO 10:50:39,159 Commit log directory: /var/lib/cassandra/commitlog
 INFO 10:50:39,160 DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
 INFO 10:50:39,160 disk_failure_policy is stop
 INFO 10:50:39,165 Global memtable threshold is enabled at 332MB
 INFO 10:50:39,298 Not using multi-threaded compaction
 INFO 10:50:39,547 Initializing key cache with capacity of 49 MBs.
 INFO 10:50:39,560 Scheduling key cache save to each 14400 seconds (going to save all keys).
 INFO 10:50:39,562 Initializing row cache with capacity of 0 MBs
 INFO 10:50:39,570 Scheduling row cache save to each 0 seconds (going to save all keys).
 INFO 10:50:39,977 Couldn't detect any schema definitions in local storage.
 INFO 10:50:39,977 To create keyspaces and column families, see 'help create keyspace' in
cqlsh.
 INFO 10:50:40,064 Enqueuing flush of Memtable-local@1164440413(149/149 serialized/live bytes,
6 ops)
 INFO 10:50:40,066 Writing Memtable-local@1164440413(149/149 serialized/live bytes, 6 ops)
 INFO 10:50:40,109 Completed flushing /var/lib/cassandra/data/system/local/system-local-ja-1-Data.db
(178 bytes) for commitlog position ReplayPosition(segmentId=1374141039946, position=417)
 INFO 10:50:40,129 No commitlog files found; skipping replay
 INFO 10:50:40,466 Cassandra version: 2.0.0-beta1-SNAPSHOT
 INFO 10:50:40,467 Thrift API version: 19.37.0
 INFO 10:50:40,473 CQL supported versions: 2.0.0,3.1.0 (default: 3.1.0)
 INFO 10:50:40,509 Loading persisted ring state
 INFO 10:50:40,512 Starting up server gossip
 WARN 10:50:40,534 No host ID found, created 8e0fac5a-b8c6-4a6b-b476-de794fb4b85c (Note: This
should happen exactly once per node).
 INFO 10:50:40,539 Enqueuing flush of Memtable-local@366873404(300/300 serialized/live bytes,
11 ops)
 INFO 10:50:40,540 Writing Memtable-local@366873404(300/300 serialized/live bytes, 11 ops)
 INFO 10:50:40,552 Completed flushing /var/lib/cassandra/data/system/local/system-local-ja-2-Data.db
(293 bytes) for commitlog position ReplayPosition(segmentId=1374141039946, position=81742)
Broadcast: /134.36.9.8
Size of seeds0
Seed : 127.0.0.1
 INFO 10:50:40,605 Starting Messaging Service on port 7000
 INFO 10:50:40,647 Enqueuing flush of Memtable-local@210156003(86/86 serialized/live bytes,
4 ops)
 INFO 10:50:40,647 Writing Memtable-local@210156003(86/86 serialized/live bytes, 4 ops)
 INFO 10:50:40,657 Completed flushing /var/lib/cassandra/data/system/local/system-local-ja-3-Data.db
(118 bytes) for commitlog position ReplayPosition(segmentId=1374141039946, position=82003)
 INFO 10:50:40,659 JOINING: waiting for ring information
 INFO 10:50:41,582 Handshaking version with /127.0.0.1
 INFO 10:50:46,585 Handshaking version with /127.0.0.1
 INFO 10:50:46,585 Cannot handshake version with /127.0.0.1
 INFO 10:50:51,586 Cannot handshake version with /127.0.0.1
 INFO 10:50:51,587 Handshaking version with /127.0.0.1
 INFO 10:50:56,589 Cannot handshake version with /127.0.0.1
 INFO 10:50:56,590 Handshaking version with /127.0.0.1
 INFO 10:51:01,591 Cannot handshake version with /127.0.0.1
 INFO 10:51:01,593 Handshaking version with /127.0.0.1
 INFO 10:51:06,594 Cannot handshake version with /127.0.0.1
 INFO 10:51:06,595 Handshaking version with /127.0.0.1
 INFO 10:51:10,713 JOINING: schema complete, ready to bootstrap
 INFO 10:51:10,716 JOINING: getting bootstrap token
 INFO 10:51:10,766 Enqueuing flush of Memtable-local@1819776360(10114/10114 serialized/live
bytes, 257 ops)
 INFO 10:51:10,767 Writing Memtable-local@1819776360(10114/10114 serialized/live bytes, 257
ops)
 INFO 10:51:10,797 Completed flushing /var/lib/cassandra/data/system/local/system-local-ja-4-Data.db
(5290 bytes) for commitlog position ReplayPosition(segmentId=1374141039946, position=94018)
 INFO 10:51:10,813 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-ja-2-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-ja-1-Data.db'), SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-ja-3-Data.db'),
SSTableReader(path='/var/lib/cassandra/data/system/local/system-local-ja-4-Data.db')]
 INFO 10:51:10,814 JOINING: sleeping 30000 ms for pending range setup
 INFO 10:51:10,864 Compacted 4 sstables to [/var/lib/cassandra/data/system/local/system-local-ja-5,].
 5,879 bytes to 5,718 (~97% of original) in 46ms = 0.118546MB/s.  4 total rows, 1 unique.
 Row merge counts were {1:0, 2:0, 3:0, 4:1, }
 INFO 10:51:11,597 Cannot handshake version with /127.0.0.1
 INFO 10:51:11,598 Handshaking version with /127.0.0.1
 INFO 10:51:16,598 Cannot handshake version with /127.0.0.1
 INFO 10:51:16,599 Handshaking version with /127.0.0.1
 INFO 10:51:21,600 Cannot handshake version with /127.0.0.1
 INFO 10:51:21,601 Handshaking version with /127.0.0.1
 INFO 10:51:26,602 Cannot handshake version with /127.0.0.1
 INFO 10:51:26,604 Handshaking version with /127.0.0.1
 INFO 10:51:31,605 Cannot handshake version with /127.0.0.1
 INFO 10:51:31,606 Handshaking version with /127.0.0.1
 INFO 10:51:36,607 Cannot handshake version with /127.0.0.1
 INFO 10:51:36,609 Handshaking version with /127.0.0.1
ERROR 10:51:40,816 Exception encountered during startup
java.lang.IllegalStateException: Unable to contact any seeds!
	at org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:897)
	at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:668)
	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:527)
	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:426)
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:354)
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:453)
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:496)
java.lang.IllegalStateException: Unable to contact any seeds!
	at org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:897)
	at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:668)
	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:527)
	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:426)
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:354)
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:453)
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:496)
Exception encountered during startup: Unable to contact any seeds!
ERROR 10:51:40,822 Exception in thread Thread[StorageServiceShutdownHook,5,main]
java.lang.NullPointerException
	at org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:310)
	at org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:359)
	at org.apache.cassandra.service.StorageService.access$000(StorageService.java:95)
	at org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:492)
	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
	at java.lang.Thread.run(Thread.java:724)

                
> If a Seed can't be contacted, a new node comes up as a cluster of 1
> -------------------------------------------------------------------
>
>                 Key: CASSANDRA-5768
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5768
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 2.0 beta 1
>            Reporter: Andy Cobley
>            Assignee: Brandon Williams
>            Priority: Minor
>             Fix For: 2.0 beta 2
>
>         Attachments: 5768.txt, cassandra.yaml
>
>
> Setting up a new test cluster using  2.0.0-beta1 and I noticed the following behaviour
with vnodes turned on.  
> I bring up one node all well and good.  however if I bring up a second node, that can't
contact the first (the first being the seed for the second) after a short period of time,
the second goes ahead and assumes it's the only node and bootstraps with all tokens.  
> NOTE also this email from Robert Coli 
> To: user@cassandra.apache.org
> Obviously if you have defined a seed and cannot contact it, the node should not start
as a cluster of one. I have a to-do list item to file a JIRA on the subject, but if you wanted
to file and link us, that'd be super. :)
> Startup trace (from the can't contact the seed messages below).
> http://aep.appspot.com/display/ABcWltCES1srzPrj5CkS69-GB8o/

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message