Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EED7110EBD for ; Tue, 31 Dec 2013 02:19:43 +0000 (UTC) Received: (qmail 53361 invoked by uid 500); 31 Dec 2013 02:19:41 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 53311 invoked by uid 500); 31 Dec 2013 02:19:41 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 53303 invoked by uid 99); 31 Dec 2013 02:19:41 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 31 Dec 2013 02:19:41 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy includes SPF record at spf.trusted-forwarder.org) Received: from [209.85.128.54] (HELO mail-qe0-f54.google.com) (209.85.128.54) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 31 Dec 2013 02:19:35 +0000 Received: by mail-qe0-f54.google.com with SMTP id cy11so12231568qeb.27 for ; Mon, 30 Dec 2013 18:19:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:content-type:message-id:mime-version :subject:date:references:to:in-reply-to; bh=Gq1pgkATBS8Hsr1S74sInN6xDqZIQbqZZOUqfYmFv1Y=; b=JZIHS/NneOk4xkWT6v9p77Is36lxDSok5iXuqpSkEhDmoESZwRxEOwsEfSw/ZzvjFM NPjhCcPCo30uutZTvlDblVCJk/f1E7gnFDkX0YBShITd8ZHT7VPIyiFBHOO5+ZIkzyA2 DRI2Eo8xrIquUEemAj+uRTAC+wQzoiFcZ4X1Rv1gBp2dUXA++nbIylu8M01PbE1rV8fz RDVFXXosPIl9rePkqfErRjSJOhTJQhta5L3pxZiTCJww80l76dLdoQ346scYrO7QDA8K lQKjO7CpPRUaNeRoOdAJuzngVtMImXUhoNJ6JZqHIpNo3W1xaGqS263vhzMy6mnGASRm iixA== X-Gm-Message-State: ALoCoQkNOdiXN4zGzdl7JHhbKRcqsZl+LLj26bGpLZkHgyOVBcFSN+EFg6AXp/m4CdS1znoo7Lwx X-Received: by 10.49.34.207 with SMTP id b15mr116196655qej.49.1388456354503; Mon, 30 Dec 2013 18:19:14 -0800 (PST) Received: from [172.16.1.18] ([203.86.207.101]) by mx.google.com with ESMTPSA id nz10sm61870605qeb.10.2013.12.30.18.19.12 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 30 Dec 2013 18:19:13 -0800 (PST) From: Aaron Morton Content-Type: multipart/alternative; boundary="Apple-Mail=_93050B6E-757F-408E-A61D-44F36856543F" Message-Id: <0410431D-4CA0-46B3-9FE6-284BB84E6C05@thelastpickle.com> Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: cassandra monitoring Date: Tue, 31 Dec 2013 15:19:05 +1300 References: To: Cassandra User In-Reply-To: X-Mailer: Apple Mail (2.1827) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_93050B6E-757F-408E-A61D-44F36856543F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 > JMX is doing it's thing on the cassandra node and is running on port = 8081 Have you set the JMX port for the cluster in Ops Centre ? The default = JMX port has been 7199 for a while. Off the top of the my head it=92s in the same area where you specify the = initial nodes in the cluster, maybe behind an =93Advanced=94 button.=20 The Ops Centre agent talks to the server to find out what JMX port it = should use to talk to the local Cassandra install.=20 Also check the logs in /var/log/datastax=20 Cheers ----------------- Aaron Morton New Zealand @aaronmorton Co-Founder & Principal Consultant Apache Cassandra Consulting http://www.thelastpickle.com On 30/12/2013, at 2:21 am, Tim Dunphy wrote: > Hi all, >=20 > I'm attempting to configure datastax agent so that opscenter can = monitor cassandra. I am running cassandra 2.0.3 and = opscenter-4.0.1-2.noarch running. Cassandra is running on a centos 5.9 = host and the opscenter host is running on centos 6.5 >=20 > A ps shows the agent running >=20 > [root@beta:~] #ps -ef | grep datastax | grep -v grep=20 > root 2166 1 0 03:31 ? 00:00:00 /bin/bash = /usr/share/datastax-agent/bin/datastax_agent_monitor > 106 2187 1 0 03:31 ? 00:01:37 = /etc/alternatives/javahome/bin/java -Xmx40M -Xms40M = -Djavax.net.ssl.trustStore=3D/var/lib/datastax-agent/ssl/agentKeyStore = -Djavax.net.ssl.keyStore=3D/var/lib/datastax-agent/ssl/agentKeyStore = -Djavax.net.ssl.keyStorePassword=3Dopscenter = -Dagent-pidfile=3D/var/run/datastax-agent/datastax-agent.pid = -Dlog4j.configuration=3D/etc/datastax-agent/log4j.properties -jar = datastax-agent-4.0.2-standalone.jar = /var/lib/datastax-agent/conf/address.yaml >=20 > And the service itself claims that it is running: >=20 > [root@beta:~] #service datastax-agent status=20 > datastax-agent (pid 2187) is running... >=20 > On the cassandra node I have ports 61620 and 61621 open on the = firewall. >=20 > But if I do an lsof and look for those ports I see no activity there. >=20 > [root@beta:~] #lsof -i :61620=20 > [root@beta:~] #lsof -i :61621 >=20 > And a netstat turns up nothing either: > [root@beta:~] #netstat -tapn | egrep "(datastax|ops)" >=20 >=20 > So I guess it should come as no surprise that the opscenter interface = reports the node as down. >=20 > And trying to reinstall the agent remotely by clicking the 'fix' link = errors out: >=20 > g is null >=20 > If you need to make changes, you can press "Retry" and the = installations will be retried. >=20 > And also I got on another attempt: >=20 > "Cannot call method 'getRequstStatus' of null.=20 >=20 > I'm really wondering why I'm doing wrong here, and how I can work my = way out of this quagmire. It would be beyond awesome to actually get = this working! >=20 > I've also attempted to get Cassandra Cluster Admin working. JMX is = doing it's thing on the cassandra node and is running on port 8081. CCA = is running on the same host as the opscenter. >=20 > But cca gives me this error once I log in: >=20 > Cassandra Cluster Admin >=20 > Logout >=20 > Fatal error: Uncaught exception 'TTransportException' with message = 'TSocket: timed out reading 4 bytes from beta.jokefire.com:9160' in = /var/www/Cassandra-Cluster-Admin/include/thrift/transport/TSocket.php:268 = Stack trace: #0 = /var/www/Cassandra-Cluster-Admin/include/thrift/transport/TTransport.php(8= 7): TSocket->read(4) #1 = /var/www/Cassandra-Cluster-Admin/include/thrift/transport/TFramedTransport= .php(135): TTransport->readAll(4) #2 = /var/www/Cassandra-Cluster-Admin/include/thrift/transport/TFramedTransport= .php(102): TFramedTransport->readFrame() #3 = /var/www/Cassandra-Cluster-Admin/include/thrift/transport/TTransport.php(8= 7): TFramedTransport->read(4) #4 = /var/www/Cassandra-Cluster-Admin/include/thrift/protocol/TBinaryProtocol.p= hp(300): TTransport->readAll(4) #5 = /var/www/Cassandra-Cluster-Admin/include/thrift/protocol/TBinaryProtocol.p= hp(192): TBinaryProtocol->readI32(NULL) #6 = /var/www/Cassandra-Cluster-Admin/include/thrift/packages/cassandra/cassand= ra.Cassandra.client.php(1017): TBinaryProtocol->readMessageBegin(NULL, = 0, 0) # in = /var/www/Cassandra-Cluster-Admin/include/thrift/transport/TSocket.php on = line 268 >=20 > Any advice I could get on my CCA problem and /or my Opcenter problem = would be great and appreciated. >=20 > Thanks > Tim >=20 > --=20 > GPG me!! >=20 > gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B >=20 --Apple-Mail=_93050B6E-757F-408E-A61D-44F36856543F Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252
 JMX = is doing it's thing on the cassandra node and is running on port = 8081
Have you set the JMX port for the cluster in Ops = Centre ? The default JMX port has been 7199 for a = while.

Off the top of the my head it=92s in the same = area where you specify the initial nodes in the cluster, maybe behind an = =93Advanced=94 button. 

The Ops Centre = agent talks to the server to find out what JMX port it should use to = talk to the local Cassandra install. 

Also = check the logs in = /var/log/datastax 

Cheers

http://www.thelastpickle.com

On 30/12/2013, at 2:21 am, Tim Dunphy <bluethundr@gmail.com> = wrote:

Hi = all,

I'm attempting to configure datastax agent so that = opscenter can monitor cassandra. I am running cassandra 2.0.3 and = opscenter-4.0.1-2.noarch running. Cassandra is running on a centos 5.9 = host and the opscenter host is running on centos 6.5

A ps shows the agent running

[root@beta:~] #ps -ef = | grep datastax | grep -v grep
root      = 2166     1  0 03:31 = ?        00:00:00 /bin/bash = /usr/share/datastax-agent/bin/datastax_agent_monitor
106       2187     = 1  0 03:31 ?        00:01:37 = /etc/alternatives/javahome/bin/java -Xmx40M -Xms40M = -Djavax.net.ssl.trustStore=3D/var/lib/datastax-agent/ssl/agentKeyStore = -Djavax.net.ssl.keyStore=3D/var/lib/datastax-agent/ssl/agentKeyStore = -Djavax.net.ssl.keyStorePassword=3Dopscenter = -Dagent-pidfile=3D/var/run/datastax-agent/datastax-agent.pid = -Dlog4j.configuration=3D/etc/datastax-agent/log4j.properties -jar = datastax-agent-4.0.2-standalone.jar = /var/lib/datastax-agent/conf/address.yaml

And the service itself claims that it is = running:

[root@beta:~] #service datastax-agent status =
datastax-agent (pid  2187) is = running...

On the cassandra node I have ports = 61620 and 61621 open on the firewall.

But if I do an lsof and look for those ports I see no activity = there.

[root@beta:~] #lsof -i :61620
[root@beta:~] #lsof -i = :61621

And a netstat turns up nothing either:
[root@beta:~] #netstat -tapn | egrep = "(datastax|ops)"


So I guess it should come as no = surprise that the opscenter interface reports the node as = down.

And trying to reinstall the agent remotely by = clicking the 'fix' link errors out:

g is null

If you need to make changes, you can press "Retry" = and the installations will be retried.

And also I got = on another attempt:

"Cannot call method = 'getRequstStatus' of null.

I'm really wondering why I'm doing wrong here, and how I = can work my way out of this quagmire. It would be beyond awesome to = actually get this working!

I've also attempted to get = Cassandra Cluster Admin working. JMX is doing it's thing on the = cassandra node and is running on port 8081. CCA is running on the same = host as the opscenter.

But cca gives me this error once I log in:

Cassandra Cluster = Admin

Logout
=09
Fatal error: Uncaught exception 'TTransportException' with=20 message 'TSocket: timed out reading 4 bytes from beta.jokefire.com:9160' in=20 = /var/www/Cassandra-Cluster-Admin/include/thrift/transport/TSocket.php:268 Stack trace: #0=20 = /var/www/Cassandra-Cluster-Admin/include/thrift/transport/TTransport.php(8= 7): TSocket->read(4) #1=20 = /var/www/Cassandra-Cluster-Admin/include/thrift/transport/TFramedTransport= .php(135): TTransport->readAll(4) #2=20 = /var/www/Cassandra-Cluster-Admin/include/thrift/transport/TFramedTransport= .php(102): TFramedTransport->readFrame() #3=20 = /var/www/Cassandra-Cluster-Admin/include/thrift/transport/TTransport.php(8= 7): TFramedTransport->read(4) #4=20 = /var/www/Cassandra-Cluster-Admin/include/thrift/protocol/TBinaryProtocol.p= hp(300): TTransport->readAll(4) #5=20 = /var/www/Cassandra-Cluster-Admin/include/thrift/protocol/TBinaryProtocol.p= hp(192): TBinaryProtocol->readI32(NULL) #6=20 = /var/www/Cassandra-Cluster-Admin/include/thrift/packages/cassandra/cassand= ra.Cassandra.client.php(1017): TBinaryProtocol->readMessageBegin(NULL, 0, 0) # in = /var/www/Cassandra-Cluster-Admin/include/thrift/transport/TSocket.php on line 268

Any advice I could get on my CCA = problem and /or my Opcenter problem would be great and appreciated.
=

Thanks
Tim

--
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B


= --Apple-Mail=_93050B6E-757F-408E-A61D-44F36856543F--