I overwrote all sstable and system files from a snapshot that I took right before the upgrade and it worked this time.

Everything is back to normal and thanks again.

On Sun, Sep 4, 2011 at 12:05 AM, Eric Czech <eric@nextbigsound.com> wrote:
I shutdown the cassandra java process, deleted the Schema and Migration tables, and then restarted the node but am now getting a "Could not reach schema agreement" error.  After running "describe cluster" from a working node, I can see that all other nodes have a consistent schema and it's only the one node that is having an issue.

Do you know why there might still be a disagreement?

Thank you for your help!

On Sat, Sep 3, 2011 at 11:40 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
That means somehow there is more than one copy of that CF declared in
the schema on that node.  Delete the schema and let it pull it from
another node, as in wiki.apache.org/cassandra/FAQ#schema_disagreement.

On Sun, Sep 4, 2011 at 12:03 AM, Eric Czech <eric@nextbigsound.com> wrote:
> Hi,
> I recently upgraded 10 nodes from 7.5 to 8.4 and 9 of them work now but on
> one node I'm getting an exception on startup that I can't seem to fix.  Has
> anyone seen this before or have any suggestions as to how to correct the
> issue here?  Here's the exception I'm getting:
> java.lang.RuntimeException: javax.management.InstanceAlreadyExistsException:
> org.apache.cassandra.db:type=ColumnFamilies,keyspace=Keyspace1,columnfamily=<CF_Name>
> at
> org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:315)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:466)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:436)
> at org.apache.cassandra.db.Table.initCf(Table.java:369)
> at org.apache.cassandra.db.Table.<init>(Table.java:306)
> at org.apache.cassandra.db.Table.open(Table.java:111)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:336)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:91)
> Caused by: javax.management.InstanceAlreadyExistsException:
> org.apache.cassandra.db:type=ColumnFamilies,keyspace=Keyspace1,columnfamily=<CF_Name>
> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:467)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1520)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:986)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:938)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:330)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:516)
> at
> org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:311)

Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support