incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin Clark <co...@clark.ws>
Subject Re: decommissioning a node
Date Sun, 25 May 2014 14:19:55 GMT
Try this:

nodetool decomission <host-id-of-node-to-decomission>

UN means UP, NORMAL

--
Colin
+1 320 221 9531



On Sun, May 25, 2014 at 9:09 AM, Tim Dunphy <bluethundr@gmail.com> wrote:

> Also for information that may help diagnose this issue I am running
> cassandra 2.0.7
>
> I am also using these java options:
>
> [root@beta:/etc/alternatives/cassandrahome] #grep -i jvm_opts
> conf/cassandra-env.sh  | grep -v '#'
> JVM_OPTS="$JVM_OPTS -ea"
>     JVM_OPTS="$JVM_OPTS -javaagent:$CASSANDRA_HOME/lib/jamm-0.2.5.jar"
> JVM_OPTS="$JVM_OPTS -XX:+CMSClassUnloadingEnabled"
> JVM_OPTS="$JVM_OPTS -XX:+UseThreadPriorities"
> JVM_OPTS="$JVM_OPTS -XX:ThreadPriorityPolicy=42"
> JVM_OPTS="$JVM_OPTS -Xms${MAX_HEAP_SIZE}"
> JVM_OPTS="$JVM_OPTS -Xmx${MAX_HEAP_SIZE}"
> JVM_OPTS="$JVM_OPTS -Xmn${HEAP_NEWSIZE}"
> JVM_OPTS="$JVM_OPTS -XX:+HeapDumpOnOutOfMemoryError"
>     JVM_OPTS="$JVM_OPTS
> -XX:HeapDumpPath=$CASSANDRA_HEAPDUMP_DIR/cassandra-`date +%s`-pid$$.hprof"
> JVM_OPTS="$JVM_OPTS -Xss256k"
> JVM_OPTS="$JVM_OPTS -XX:StringTableSize=1000003"
> JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC"
> JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC"
> JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled"
> JVM_OPTS="$JVM_OPTS -XX:SurvivorRatio=8"
> JVM_OPTS="$JVM_OPTS -XX:MaxTenuringThreshold=1"
> JVM_OPTS="$JVM_OPTS -XX:CMSInitiatingOccupancyFraction=75"
> JVM_OPTS="$JVM_OPTS -XX:+UseCMSInitiatingOccupancyOnly"
> JVM_OPTS="$JVM_OPTS -XX:+UseTLAB"
>     JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark"
> JVM_OPTS="$JVM_OPTS -Djava.net.preferIPv4Stack=true"
> JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT"
> JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.ssl=false"
> JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.authenticate=false"
> JVM_OPTS="$JVM_OPTS $JVM_EXTRA_OPTS"
>
>
> Still need to figure out why the node I want to decommission isn't
> listening on port 7199 and how I can actually decommission it.
>
> Thanks
> Tim
>
>
> On Sun, May 25, 2014 at 9:20 AM, Tim Dunphy <bluethundr@gmail.com> wrote:
>
>>
>> Hey all,
>>
>> I'm attempting to decommission a node I want to remove.
>>
>> First I get a status of the ring
>>
>> [root@beta-new:~] #nodetool status
>>
>> Datacenter: datacenter1
>>
>> =======================
>>
>> Status=Up/Down
>>
>> |/ State=Normal/Leaving/Joining/Moving
>>
>> --  Address         Load       Tokens  Owns   Host ID
>>           Rack
>>
>> UN  10.10.1.94  197.37 KB  256     49.4%
>> fd2f76ae-8dcf-4e93-a37f-bf1e9088696e  rack1
>>
>> UN  10.10.1.18    216.95 KB  256     50.6%
>> f2a48fc7-a362-43f5-9061-4bb3739fdeaf  rack
>>
>>
>> I see that the node I want to remove is UP. Tho I believe UN means up I
>> don't know what it stands for.
>>
>>
>> [root@beta-new:~] #nodetool -host  10.10.1.18 decommission
>>
>> Failed to connect to ' 10.10.1.18 : Connection timed out
>>
>> The connection to the node I want to decommission times out. :(
>>
>> I’m running this node from the seed node, and while I do see port 7199
>> active and listening there, I do NOT see this port active and listening on
>> the node that I want to decommission.
>>
>>
>> Seed node:
>>
>> [root@beta-new:~] #lsof -i :7199
>>
>> COMMAND   PID USER   FD   TYPE    DEVICE SIZE/OFF NODE NAME
>>
>> java    15331 root   51u  IPv4 566368606      0t0  TCP *:7199 (LISTEN)
>>
>>
>> [root@beta:/etc/alternatives/cassandrahome] #lsof -i :7199
>>
>> [root@beta:/etc/alternatives/cassandrahome] #
>>
>>
>> However cassandra does seem to be running on the node I want to
>> decommission in addition to it being shown as UN by nodetool status:
>>
>>
>> [root@beta:/etc/alternatives/cassandrahome] #netstat -tulpn | grep -i
>> listen | grep java
>>
>> tcp        0      0 0.0.0.0:46755               0.0.0.0:*
>>     LISTEN      23039/java
>>
>> tcp        0      0 10.10.1.18:9160           0.0.0.0:*
>>   LISTEN      23039/java
>>
>> tcp        0      0 0.0.0.0:42990               0.0.0.0:*
>>     LISTEN      23039/java
>>
>> tcp        0      0 10.10.1.18:8081           0.0.0.0:*
>>   LISTEN      23039/java
>>
>> tcp        0      0 10.10.1.18:9042           0.0.0.0:*
>>   LISTEN      23039/java
>>
>> tcp        0      0 10.10.1.18:7000           0.0.0.0:*
>>   LISTEN      23039/java
>>
>> tcp        0      0 0.0.0.0:7198                0.0.0.0:*
>>     LISTEN      23039/java
>>
>>
>> So why do you think my seed is listening on port 7199 but the node I want
>> to get rid of is not? And how can I accomplish my goal of deleting the
>> unwanted node?
>>
>>
>> Thanks
>>
>> Tim
>>
>>
>>
>> --
>> GPG me!!
>>
>> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
>>
>>
>
>
> --
> GPG me!!
>
> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
>
>

Mime
View raw message