cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Giovanni Usai <giovanni.u...@francelabs.com>
Subject Re: Best way to get Cassandra status in Bash
Date Thu, 07 Jan 2016 10:52:45 GMT
Hello,
thanks a lot!
The solution proposed by Gerard works perfectly.

This is the snippet of what I have done:

     echo "Checking if Cassandra is up and running ..."
     # Try to connect on Cassandra's JMX port 7199
     nc -z localhost 7199
     nc_return=$?

     # Try to connect on Cassandra CQLSH port 9042
     nc -z localhost 9042
     let "cassandra_status = nc_return + $?"

     retries=1
     while (( retries < 6 && cassandra_status != 0 )); do
         echo "Cassandra doesn't reply to requests on ports 7199 and/or 
9042. Sleeping for a while and trying again... retry ${retries}"

         # Sleep for a while
         sleep 2s

         # Try again to connect to Cassandra
         echo "Checking if Cassandra is up and running ..."
         nc -z localhost 7199
         nc_return=$?

         nc -z localhost 9042
         let "cassandra_status = nc_return + $?"

         let "retries++"
     done

     if [ $cassandra_status -ne 0 ]; then
         echo "/!\ ERROR: Cassandra startup has ended with errors; 
please check log file ${DATAFARI_LOGS}/cassandra-startup.log"
     else
         echo "Cassandra startup completed successfully --- OK"
         $CASSANDRA_HOME/bin/cqlsh -f 
$DATAFARI_HOME/bin/common/config/cassandra/tables
     fi

Best regards,
*Giovanni Usai
* giovanni.usai@francelabs.com <mailto:giovanni.usai@francelabs.com>


www.francelabs.com <http://www.francelabs.com/>

CEEI Nice Premium
1 Bd. Maître Maurice Slama
06200 Nice FRANCE

Ph: +33 (0)9 72 43 72 85

On 01/05/2016 05:17 PM, Giovanni Usai wrote:
> Hello,
>
> thanks to everyone for the fast replies!
>
> Unfortunately, since yesterday afternoon I have been assigned to a 
> more urgent task, so I will implement the solutions you proposed in 
> the spare time and I will let you know the outcomes asap (hopefully in 
> few weeks).
>
> Thanks a lot again!
>
> Best regards,
> *Giovanni Usai
> * giovanni.usai@francelabs.com
>
>
> www.francelabs.com <http://www.francelabs.com/>
>
> CEEI Nice Premium
> 1 Bd. Maître Maurice Slama
> 06200 Nice FRANCE
>
> Ph: +33 (0)9 72 43 72 85
>
> On 01/04/2016 05:52 PM, Gerard Maas wrote:
>> Hi Giovanni,
>>
>> You could use netcat (nc) to test that the cassandra port is up and 
>> use a timeout to decide when to take an action
>>
>> nc -z localhost 9160
>>
>> check for the exit code to decide what action to take.
>>
>> -kr, Gerard.
>>
>>
>> On Mon, Jan 4, 2016 at 4:56 PM, Giovanni Usai 
>> <giovanni.usai@francelabs.com <mailto:giovanni.usai@francelabs.com>>

>> wrote:
>>
>>     Hello Gerard,
>>     thanks for your reply.
>>
>>     It seems nodetool works only when the cluster is up and running.
>>     In case of a bad startup of Cassandra, if I run "nodetool status"
>>     I get one of these 2 errors:
>>
>>     1) error: No nodes present in the cluster. Has this node finished
>>     starting up?
>>     -- StackTrace --
>>     java.lang.RuntimeException: No nodes present in the cluster. Has
>>     this node finished starting up?
>>         at
>>     org.apache.cassandra.dht.Murmur3Partitioner.describeOwnership(Murmur3Partitioner.java:129)
>>         at
>>     org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:3960)
>>         at
>>     org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:176)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>>     sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at
>>     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>
>>     2) nodetool: Failed to connect to '127.0.0.1:7199
>>     <http://127.0.0.1:7199>' - ConnectException: 'Connection refused'.
>>
>>     In both cases, the nodetool status command kills my Bash script.
>>
>>     Since I want to do some updates on Cassandra right after startup,
>>     I must wait until the cluster is ready to process requests.
>>     Depending on the hardware, the Cassandra startup may take some
>>     time, so I need to be able to detect when Cassandra is up and
>>     running.
>>
>>     What I am trying to do is something like follows:
>>
>>     execute cassandra start [=> $CASSANDRA_HOME/bin/cassandra -p
>>     $CASSANDRA_PID_FILE]
>>     while (cassandra_status != OK && retries < N){
>>         cassandra_status = some command that returns the status of
>>     cassandra startup
>>         retries++
>>     }
>>     if (cassandra_status != OK){
>>         echo the user and do some countermeasures
>>     } else {
>>         make updates on Cassandra [=> $CASSANDRA_HOME/bin/cqlsh -f
>>     $DATAFARI_HOME/bin/common/config/cassandra/tables]
>>     }
>>
>>     Do you have any idea about the command to use here?
>>     cassandra_status = some command that returns the status of
>>     cassandra startup
>>
>>     Thanks
>>
>>     Best regards,
>>     *Giovanni Usai
>>     * giovanni.usai@francelabs.com <mailto:giovanni.usai@francelabs.com>
>>
>>
>>     www.francelabs.com <http://www.francelabs.com/>
>>
>>     CEEI Nice Premium
>>     1 Bd. Maître Maurice Slama
>>     06200 Nice FRANCE
>>
>>     Ph: +33 (0)9 72 43 72 85
>>
>>     On 01/04/2016 03:51 PM, Gerard Maas wrote:
>>>     (Hit enter too fast)
>>>
>>>     In particular, `nodetool status` will give you a summary of the
>>>     status of the cluster. See the documentation for the parameters
>>>     it takes.
>>>
>>>     -kr, Gerard.
>>>
>>>     On Mon, Jan 4, 2016 at 3:49 PM, Gerard Maas
>>>     <gerard.maas@gmail.com> wrote:
>>>
>>>         I think you are looking for the nodetool utility:
>>>         https://docs.datastax.com/en/cassandra/2.1/cassandra/tools/toolsNodetool_r.html
>>>
>>>         On Mon, Jan 4, 2016 at 1:47 PM, Giovanni Usai
>>>         <giovanni.usai@francelabs.com> wrote:
>>>
>>>             Hello,
>>>             I would gladly welcome the help of the community on the
>>>             following issue I am having while starting Cassandra.
>>>
>>>             I am starting Cassandra by a Bash script in this way:
>>>
>>>             - $CASSANDRA_HOME/bin/cassandra -p $CASSANDRA_PID_FILE
>>>             and then, I submit some updates via
>>>             - $CASSANDRA_HOME/bin/cqlsh -f
>>>             $DATAFARI_HOME/bin/common/config/cassandra/tables
>>>
>>>             => First question: is it a good idea? Or, are there
>>>             better ways to do start Cassandra?
>>>
>>>
>>>             If it is a good idea to use Bash, this is my need: when
>>>             something goes wrong (e.g. privileges issue on 
>>>             Cassandra's data directory, etc), I would like to detect
>>>             it to be able to apply some countermeasures.
>>>
>>>             => Second question: do you know what's the best way to
>>>             get the Cassandra and CQLSH status from Bash (if it is
>>>             possible)?
>>>
>>>             These are all the approaches that I have already tried,
>>>             with no chance:
>>>             - use the return code of Cassandra script ($? Bash
>>>             operator), but it returns all the times 0 even if
>>>             something goes wrong.
>>>             - grep the Cassandra logs looking for "Exception" or
>>>             "Error" (after redirection with &> ), but it doesn't
>>>             work as they are not yet flushed by Cassandra (as a
>>>             result, the calling bash exits).
>>>             - get the output of the Cassandra script with something
>>>             like cassandra_return=$($CASSANDRA_HOME/bin/cassandra -p
>>>             $CASSANDRA_PID_FILE), but the variable is empty all the
>>>             times.
>>>             - detach Cassandra process from calling script with '&'
>>>             operator and then grep the logs or get the return code,
>>>             but it doesn't work neither.
>>>
>>>             Furthermore, when CQLSH script cannot connect to
>>>             Cassandra, it prints this error on console:
>>>             Connection error: ('Unable to connect to any servers',
>>>             {'127.0.0.1': error(111, "Tried connecting to
>>>             [('127.0.0.1', 9042)]. Last error: Connection refused")})
>>>             and then kills the calling script.
>>>
>>>             Just for your information, we are embedding Cassandra in
>>>             our open source product "Datafari".
>>>
>>>             Thanks for your help and
>>>             -- 
>>>
>>>             Best regards,
>>>             *Giovanni Usai
>>>             * giovanni.usai@francelabs.com
>>>
>>>
>>>             www.francelabs.com <http://www.francelabs.com/>
>>>
>>>             CEEI Nice Premium
>>>             1 Bd. Maître Maurice Slama
>>>             06200 Nice FRANCE
>>>
>>>             Ph: +33 (0)9 72 43 72 85
>>>
>>>
>>>
>>
>>
>


Mime
View raw message