Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 60EE298D0 for ; Thu, 22 Dec 2011 16:16:46 +0000 (UTC) Received: (qmail 17469 invoked by uid 500); 22 Dec 2011 16:16:46 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 17443 invoked by uid 500); 22 Dec 2011 16:16:46 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 17435 invoked by uid 99); 22 Dec 2011 16:16:46 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 22 Dec 2011 16:16:46 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 22 Dec 2011 16:16:41 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id EC0062388C22 for ; Thu, 22 Dec 2011 16:16:19 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1222321 - in /cassandra/trunk: ./ contrib/ debian/ interface/thrift/gen-java/org/apache/cassandra/thrift/ pylib/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/index/ src/java/org/apache/cassandra/tools/ Date: Thu, 22 Dec 2011 16:16:19 -0000 To: commits@cassandra.apache.org From: jake@apache.org X-Mailer: svnmailer-1.0.8-patched Message-Id: <20111222161619.EC0062388C22@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: jake Date: Thu Dec 22 16:16:18 2011 New Revision: 1222321 URL: http://svn.apache.org/viewvc?rev=1222321&view=rev Log: merge from 1.0 Added: cassandra/trunk/debian/pyversions - copied unchanged from r1222319, cassandra/branches/cassandra-1.0/debian/pyversions cassandra/trunk/pylib/setup.py - copied unchanged from r1222319, cassandra/branches/cassandra-1.0/pylib/setup.py Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/debian/cassandra.install cassandra/trunk/debian/control cassandra/trunk/debian/rules cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java cassandra/trunk/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java cassandra/trunk/src/java/org/apache/cassandra/tools/NodeCmd.java Propchange: cassandra/trunk/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 16:16:18 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 -/cassandra/branches/cassandra-1.0:1167085-1221481 +/cassandra/branches/cassandra-1.0:1167085-1222319 /cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/branches/cassandra-1.0.5:1208016 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1222321&r1=1222320&r2=1222321&view=diff ============================================================================== --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Thu Dec 22 16:16:18 2011 @@ -38,6 +38,7 @@ (CASSANDRA-3335) Merged from 0.8: * prevent new nodes from thinking down nodes are up forever (CASSANDRA-3626) + * Flush non-cfs backed secondary indexes (CASSANDRA-3659) 1.0.6 Propchange: cassandra/trunk/contrib/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 16:16:18 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 -/cassandra/branches/cassandra-1.0/contrib:1167085-1221481 +/cassandra/branches/cassandra-1.0/contrib:1167085-1222319 /cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/branches/cassandra-1.0.5/contrib:1208016 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Modified: cassandra/trunk/debian/cassandra.install URL: http://svn.apache.org/viewvc/cassandra/trunk/debian/cassandra.install?rev=1222321&r1=1222320&r2=1222321&view=diff ============================================================================== --- cassandra/trunk/debian/cassandra.install (original) +++ cassandra/trunk/debian/cassandra.install Thu Dec 22 16:16:18 2011 @@ -11,5 +11,6 @@ bin/json2sstable usr/bin bin/sstable2json usr/bin bin/sstablekeys usr/bin bin/sstableloader usr/bin +bin/cqlsh usr/bin lib/*.jar usr/share/cassandra/lib lib/licenses usr/share/doc/cassandra Modified: cassandra/trunk/debian/control URL: http://svn.apache.org/viewvc/cassandra/trunk/debian/control?rev=1222321&r1=1222320&r2=1222321&view=diff ============================================================================== --- cassandra/trunk/debian/control (original) +++ cassandra/trunk/debian/control Thu Dec 22 16:16:18 2011 @@ -2,7 +2,8 @@ Source: cassandra Section: misc Priority: extra Maintainer: Eric Evans -Build-Depends: debhelper (>= 5), openjdk-6-jdk (>= 6b11) | java6-sdk, ant (>= 1.7), ant-optional (>= 1.7), subversion +Uploaders: Sylvain Lebresne +Build-Depends: debhelper (>= 5), openjdk-6-jdk (>= 6b11) | java6-sdk, ant (>= 1.7), ant-optional (>= 1.7), subversion, python-support (>= 0.6) Homepage: http://cassandra.apache.org Vcs-Svn: https://svn.apache.org/repos/asf/cassandra/trunk Vcs-Browser: http://svn.apache.org/viewvc/cassandra/trunk @@ -10,7 +11,7 @@ Standards-Version: 3.8.3 Package: cassandra Architecture: all -Depends: openjdk-6-jre-headless (>= 6b11) | java6-runtime, jsvc (>= 1.0), libcommons-daemon-java (>= 1.0), adduser, libjna-java +Depends: openjdk-6-jre-headless (>= 6b11) | java6-runtime, jsvc (>= 1.0), libcommons-daemon-java (>= 1.0), adduser, libjna-java, python-cql, ${python:Depends}, ${misc:Depends} Conflicts: apache-cassandra1 Replaces: apache-cassandra1 Description: distributed storage system for structured data Modified: cassandra/trunk/debian/rules URL: http://svn.apache.org/viewvc/cassandra/trunk/debian/rules?rev=1222321&r1=1222320&r2=1222321&view=diff ============================================================================== --- cassandra/trunk/debian/rules (original) +++ cassandra/trunk/debian/rules Thu Dec 22 16:16:18 2011 @@ -18,6 +18,7 @@ clean: rm -rf bin/java find -name "*.pyc" -exec rm '{}' ';' find -name "*py.class" -exec rm '{}' ';' + $(RM) -r pylib/build dh_clean build: build-stamp @@ -26,6 +27,8 @@ build-stamp: printf "version=%s" $(VERSION) > build.properties $(ANT) jar + cd pylib && python setup.py install --no-compile --install-layout deb \ + --root $(CURDIR)/debian/cassandra touch build-stamp @@ -34,6 +37,7 @@ install: build dh_testroot dh_installdirs dh_install + dh_pysupport # Copy in the jar and symlink to something stable dh_install build/apache-cassandra-$(VERSION).jar \ Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 16:16:18 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018 -/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167085-1221481 +/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167085-1222319 /cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/branches/cassandra-1.0.5/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1208016 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 16:16:18 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1101014-1125018 -/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1167085-1221481 +/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1167085-1222319 /cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/branches/cassandra-1.0.5/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1208016 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 16:16:18 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1101014-1125018 -/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1167085-1221481 +/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1167085-1222319 /cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/branches/cassandra-1.0.5/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1208016 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 16:16:18 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1101014-1125018 -/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1167085-1221481 +/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1167085-1222319 /cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/branches/cassandra-1.0.5/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1208016 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Dec 22 16:16:18 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1090934-1125013,1125019-1198724,1198726-1206097,1206099-1212854,1212938,1214916 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1101014-1125018 -/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1167085-1221481 +/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1167085-1222319 /cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/branches/cassandra-1.0.5/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1208016 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java:1051699-1053689 Modified: cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1222321&r1=1222320&r2=1222321&view=diff ============================================================================== --- cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java Thu Dec 22 16:16:18 2011 @@ -47,6 +47,7 @@ import org.apache.cassandra.db.compactio import org.apache.cassandra.db.filter.IFilter; import org.apache.cassandra.db.filter.QueryFilter; import org.apache.cassandra.db.filter.QueryPath; +import org.apache.cassandra.db.index.SecondaryIndex; import org.apache.cassandra.db.index.SecondaryIndexManager; import org.apache.cassandra.db.marshal.AbstractType; import org.apache.cassandra.dht.*; @@ -669,7 +670,7 @@ public class ColumnFamilyStore implement logger.debug("flush position is {}", ctx); // submit the memtable for any indexed sub-cfses, and our own. - List icc = new ArrayList(); + final List icc = new ArrayList(); // don't assume that this.memtable is dirty; forceFlush can bring us here during index build even if it is not for (ColumnFamilyStore cfs : concatWithIndexes()) { @@ -701,6 +702,19 @@ public class ColumnFamilyStore implement public void runMayThrow() throws InterruptedException, IOException { latch.await(); + + if (!icc.isEmpty()) + { + //only valid when memtables exist + + for (SecondaryIndex index : indexManager.getIndexesNotBackedByCfs()) + { + // flush any non-cfs backed indexes + logger.info("Flushing SecondaryIndex {}", index); + index.forceBlockingFlush(); + } + } + if (writeCommitLog) { // if we're not writing to the commit log, we are replaying the log, so marking Modified: cassandra/trunk/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java?rev=1222321&r1=1222320&r2=1222321&view=diff ============================================================================== --- cassandra/trunk/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java Thu Dec 22 16:16:18 2011 @@ -308,7 +308,27 @@ public class SecondaryIndexManager return cfsList; } - + /** + * @return all indexes which do *not* use a backing CFS internally + */ + public Collection getIndexesNotBackedByCfs() + { + // we use identity map because per row indexes use same instance + // across many columns + IdentityHashMap indexList = new IdentityHashMap(); + + for (Map.Entry entry : indexesByColumn.entrySet()) + { + ColumnFamilyStore cfs = entry.getValue().getIndexCfs(); + + if (cfs == null) + indexList.put(entry.getValue(), null); + } + + return indexList.keySet(); + } + + /** * Removes obsolete index entries and creates new ones for the given row key * and mutated columns. Modified: cassandra/trunk/src/java/org/apache/cassandra/tools/NodeCmd.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/tools/NodeCmd.java?rev=1222321&r1=1222320&r2=1222321&view=diff ============================================================================== --- cassandra/trunk/src/java/org/apache/cassandra/tools/NodeCmd.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/tools/NodeCmd.java Thu Dec 22 16:16:18 2011 @@ -158,7 +158,7 @@ public class NodeCmd addCmdHelp(header, "cleanup [keyspace] [cfnames]", "Run cleanup on one or more column family"); addCmdHelp(header, "compact [keyspace] [cfnames]", "Force a (major) compaction on one or more column family"); addCmdHelp(header, "scrub [keyspace] [cfnames]", "Scrub (rebuild sstables for) one or more column family"); - addCmdHelp(header, "upgradesstables [keyspace] [cfnames]", "Scrub (rebuild sstables for) one or more column family"); + addCmdHelp(header, "upgradesstables [keyspace] [cfnames]", "Upgrade sstables for one or more column family"); addCmdHelp(header, "invalidatekeycache [keyspace] [cfnames]", "Invalidate the key cache of one or more column family"); addCmdHelp(header, "invalidaterowcache [keyspace] [cfnames]", "Invalidate the key cache of one or more column family"); addCmdHelp(header, "getcompactionthreshold ", "Print min and max compaction thresholds for a given column family");