Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9598C1017C for ; Fri, 6 Dec 2013 21:47:45 +0000 (UTC) Received: (qmail 29627 invoked by uid 500); 6 Dec 2013 21:47:42 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 29603 invoked by uid 500); 6 Dec 2013 21:47:42 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 29595 invoked by uid 99); 6 Dec 2013 21:47:42 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Dec 2013 21:47:42 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_MED,SPF_PASS,T_REMOTE_IMAGE X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy includes SPF record at spf.trusted-forwarder.org) Received: from [74.125.149.211] (HELO na3sys009aog114.obsmtp.com) (74.125.149.211) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 06 Dec 2013 21:47:36 +0000 Received: from mail-qe0-f48.google.com ([209.85.128.48]) (using TLSv1) by na3sys009aob114.postini.com ([74.125.148.12]) with SMTP ID DSNKUqJF4TF12MmAUjsA3rNjI4+kL7ab/yIS@postini.com; Fri, 06 Dec 2013 13:47:15 PST Received: by mail-qe0-f48.google.com with SMTP id gc15so1012550qeb.7 for ; Fri, 06 Dec 2013 13:47:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=uG6axYJBbM/Xf+Fhm+OBiJ+1+W69UAWDUELh8hj1asI=; b=FrN4ZQuem95WUQXzXZyB84rbKTtNgGrInDlqNLwn6Hx9h0ZGSkgkFJKMr69zBZ5QoB Cx/SbAg3qz0o3idgLm/AYIMHF7WIDRJYG+66Bx6uOO5fLeEWDbI3cmckYqQIM7o9d7SF ontrr+OBINunOhkahcXp9+Up43QDfDICjI2pimaG7lEXIAHU63X0ptRQr9YNQlUYsWPX h+Ov5hMRj7oS4B4hmakYIE1hY07gHW/WqrXtyn2EGRdHTpdnvrl9MDGug+gD9XKj+dOe /GKIYDBRoxjgR3zeMvKxA7qVmIOKu3CING9kM9WPJqt2XbFA8YvH9IpyV8mBLXyDHvfB v92w== X-Gm-Message-State: ALoCoQnNRuUb/MCyQGh/34OcxdEjNhj9nsS/EWFPpZSuL9W+ifEtS6ZskVv19gYblGRa/SFS7qpabeVukbsLcbTbIcpDxDE+Ic6Bm8p0rE54X6Yxh8XRNJsAk/z3QT+BEjkpufgzL5XsKKbeLBE0ys51IpF2buCrlTDRK+l/VnyygWxCJMDpExA= X-Received: by 10.49.15.227 with SMTP id a3mr10465775qed.66.1386366433144; Fri, 06 Dec 2013 13:47:13 -0800 (PST) X-Received: by 10.49.15.227 with SMTP id a3mr10465759qed.66.1386366432979; Fri, 06 Dec 2013 13:47:12 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.26.83 with HTTP; Fri, 6 Dec 2013 13:46:51 -0800 (PST) In-Reply-To: References: From: Franc Carter Date: Sat, 7 Dec 2013 08:46:51 +1100 Message-ID: Subject: Re: datastax community ami -- broken? --- datastax-agent conflicts with opscenter-agent To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=047d7bdc0c1ac45bc704ece49682 X-Virus-Checked: Checked by ClamAV on apache.org --047d7bdc0c1ac45bc704ece49682 Content-Type: text/plain; charset=ISO-8859-1 Hi Joaquin, A quick word of praise - addressing the issue so quickly presents a really good view of Datastax cheers On Sat, Dec 7, 2013 at 8:14 AM, Joaquin Casares wrote: > Hello again John, > > The AMI has been patched and tested for both DSE and C* and works for the > standard 3 node test. The new code has been pushed to the 2.4 branch so > launching a new set of instances will give you an updated AMI. > > You should now have the newest version of OpsCenter installed, along with > the new DataStax Agents (that replace the OpsCenter Agents). Also, I've > patched the two bugs for the motd and for allowing the other nodes to join. > > The issue came from a new release of nodetool that contained some > unexpected text that choked up the AMI code as it waited for nodes to come > online. > > Let me know if you see any further issues. > > Thanks, > > Joaquin Casares > DataStax > Software Engineer in Test > > > > > > > On Fri, Dec 6, 2013 at 2:02 PM, Joaquin Casares wrote: > >> Hey John, >> >> Thanks for letting us know. I'm also seeing that the motd gets stuck, but >> if I ctrl-c during the message and try a `nodetool status` there doesn't >> appear to be an issue. >> >> I'm currently investigating why it's getting stuck. Are you seeing >> something similar? >> >> What happens if you try to run a `sudo service cassandra restart`? Could >> you send me your /var/log/cassandra/system.log if this still fails? >> >> Also, I realize now that the package name for the newest version of >> OpsCenter changed from opscenter-free to opscenter. I committed that change >> to our dev AMI and am testing it now. Once this change is made you will no >> longer have to install agents via OpsCenter since they should already be on >> the system. That being said, you won't hit the current OpsCenter/DataStax >> Agent version mismatch you've been hitting. >> >> Also, we currently only have one AMI. Each time an instance is launched >> the newest version of the code is pulled down from >> https://github.com/riptano/ComboAMI to ensure the code never gets stale >> and can easily keep up with DSE/C* releases as well as AMI code fixes. >> >> I'll reply again as soon as I figure out and patch this motd issue. >> >> Thanks, >> >> Joaquin Casares >> DataStax >> Software Engineer in Test >> >> >> >> >> >> >> On Fri, Dec 6, 2013 at 7:16 AM, John R. Frank wrote: >> >>> Hi C* experts, >>> >>> In the last 18hrs or so, I have been having trouble getting cassandra >>> instances to launch using the datastax community AMI. Has anyone else seen >>> this? >>> >>> The instance comes up but then cassandra fails to run. The most >>> informative error message that I've seen so far is in the obscenter agent >>> install log (below) --- see especially this line: >>> >>> datastax-agent conflicts with opscenter-agent >>> >>> I have also attached the ami.log >>> >>> I run into these issues when launching with either the browser page: >>> https://aws.amazon.com/amis/datastax-auto-clustering-ami-2-2 >>> >>> Or command line, like this: >>> >>> ec2-run-instances ami-814ec2e8 -t m1.large --region us-east-1 --key >>> buildbot -n $num_instances -d "--clustername Foo --totalnodes >>> $num_instances --version community --opscenter yes" >>> >>> >>> Is there a newer AMI? >>> >>> >>> Any advice? >>> jrf >>> >>> >>> >>> >>> Some agent installations failed: >>> >>> - 10.6.133.241: Failure installing agent on 10.6.133.241. >>> Error output: >>> Unable to install the opscenter-agent package. Please check your apt-get >>> configuration as well as the agent install log (/var/log/opscenter-agent/ >>> installer.log). >>> >>> Standard output: >>> Removing old opscenter-agent files. >>> opscenter-agent: unrecognized service >>> Reading package lists... >>> Building dependency tree... >>> Reading state information... >>> 0 upgraded, 0 newly installed, 0 to remove and 171 not upgraded. >>> Starting agent installation process for version 3.2.2 >>> Reading package lists... >>> Building dependency tree... >>> Reading state information... >>> sysstat is already the newest version. >>> 0 upgraded, 0 newly installed, 0 to remove and 171 not upgraded. >>> Selecting previously unselected package opscenter-agent. >>> dpkg: regarding .../opscenter-agent.deb containing opscenter-agent: >>> datastax-agent conflicts with opscenter-agent >>> opscenter-agent (version 3.2.2) is to be installed. >>> opscenter-agent provides opscenter-agent and is to be installed. >>> dpkg: error processing opscenter_agent_setup.vYRzL0Tevn/opscenter-agent.deb >>> (--install): >>> conflicting packages - not installing opscenter-agent >>> Errors were encountered while processing: >>> opscenter_agent_setup.vYRzL0Tevn/opscenter-agent.deb >>> FAILURE: Unable to install the opscenter-agent package. Please check >>> your apt-get configuration as well as the agent install log >>> (/var/log/opscenter-agent/installer.log). >>> >>> Exit code: 1 >>> >> >> > -- *Franc Carter* | Systems architect | Sirca Ltd franc.carter@sirca.org.au | www.sirca.org.au Tel: +61 2 8355 2514 Level 4, 55 Harrington St, The Rocks NSW 2000 PO Box H58, Australia Square, Sydney NSW 1215 --047d7bdc0c1ac45bc704ece49682 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

Hi Joaquin,

A quick = word of praise - addressing the issue so quickly presents a really good vie= w of Datastax

cheers

<= br>
On Sat, Dec 7, 2013 at 8:14 AM, Joaquin Casa= res <joaquin@datastax.com> wrote:
Hello again John,

The AMI has been patc= hed and tested for both DSE and C* and works for the standard 3 node test. = The new code has been pushed to the 2.4 branch so launching a new set of in= stances will give you an updated AMI.

You should now have the newest version of OpsCenter ins= talled, along with the new DataStax Agents (that replace the OpsCenter Agen= ts). Also, I've patched the two bugs for the motd and for allowing the = other nodes to join.

The issue came from a new release of nodetool that cont= ained some unexpected text that choked up the AMI code as it waited for nod= es to come online.

Let me know if you see any furt= her issues.

Thanks,

Joaquin Casares
DataSta= x
Software Engineer in Test





On Fri, Dec= 6, 2013 at 2:02 PM, Joaquin Casares <joaquin@datastax.com> wrote:
Hey John,

Thanks for letting us know. I= 'm also seeing that the motd gets stuck, but if I ctrl-c during the mes= sage and try a `nodetool status` there doesn't appear to be an issue.

I'm currently investigating why it's getting st= uck. Are you seeing something similar?

What happen= s if you try to run a `sudo service cassandra restart`? Could you send me y= our /var/log/cassandra/system.log if this still fails?

Also, I realize now that the package name for the newes= t version of OpsCenter changed from opscenter-free to opscenter. I committe= d that change to our dev AMI and am testing it now. Once this change is mad= e you will no longer have to install agents via OpsCenter since they should= already be on the system. That being said, you won't hit the current O= psCenter/DataStax Agent version mismatch you've been hitting.

Also, we currently only have one AMI. Each time an inst= ance is launched the newest version of the code is pulled down from=A0https://github= .com/riptano/ComboAMI to ensure the code never gets stale and can easil= y keep up with DSE/C* releases as well as AMI code fixes.

I'll reply again as soon as I figure out and patch = this motd issue.

Thanks,

Joaquin Casares DataStax
Software Engineer in Test





On Fri, Dec 6, 2013 at 7= :16 AM, John R. Frank <jrf@mit.edu> wrote:
Hi C* experts,

In the last 18hrs or so, I have been having trouble getting cassandra insta= nces to launch using the datastax community AMI. =A0Has anyone else seen th= is?

The instance comes up but then cassandra fails to run. =A0The most informat= ive error message that I've seen so far is in the obscenter agent insta= ll log (below) --- see especially this line:

=A0 =A0 =A0 =A0 datastax-agent conflicts with opscenter-agent

I have also attached the ami.log

I run into these issues when launching with either the browser page:
https://aws.amazon.com/amis/datastax-auto-clustering= -ami-2-2

Or command line, like this:

ec2-run-instances ami-814ec2e8 -t m1.large --region us-east-1 --key buildbo= t -n $num_instances -d "--clustername Foo --totalnodes $num_instances = =A0--version community =A0--opscenter yes"


Is there a newer AMI?


Any advice?
jrf




Some agent installations failed:

- 10.6.133.241: Failu= re installing agent on 10.6.133.241.
Error output:
Unable to install the opscenter-agent package. Please check your apt-get co= nfiguration as well as the agent install log (/var/log/opscenter-agent/<= /u>installer.log).

Standard output:
Removing old opscenter-agent files.
opscenter-agent: unrecognized service
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 171 not upgraded.
Starting agent installation process for version 3.2.2
Reading package lists...
Building dependency tree...
Reading state information...
sysstat is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 171 not upgraded.
Selecting previously unselected package opscenter-agent.
dpkg: regarding .../opscenter-agent.deb containing opscenter-agent:
datastax-agent conflicts with opscenter-agent
opscenter-agent (version 3.2.2) is to be installed.
opscenter-agent provides opscenter-agent and is to be installed.
dpkg: error processing opscenter_agent_setup.vYRzL0Tevn/opscenter-ag= ent.deb (--install):
conflicting packages - not installing opscenter-agent
Errors were encountered while processing:
opscenter_agent_setup.vYRzL0Tevn/opscenter-agent.deb
FAILURE: Unable to install the opscenter-agent package. Please check your a= pt-get configuration as well as the agent install log (/var/log/opscenter-a= gent/installer.log).

Exit code: 1





--

Franc Carter<= /b> |<= /span> Systems architect | Sirca Ltd

franc.carter@sirca.org.au=A0|=A0www.sirca.org.au

Tel:= =A0+61 2 8355 2514

Level 4, 55 Harrington St, The Rocks NSW 2000

PO Box H58, Australia Square, Sydney NSW 1215<= /span>


--047d7bdc0c1ac45bc704ece49682--