cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tylerho...@apache.org
Subject [2/2] cassandra git commit: Merge branch 'cassandra-3.0' into trunk
Date Thu, 21 Apr 2016 21:18:57 GMT
Merge branch 'cassandra-3.0' into trunk

Conflicts:
	CHANGES.txt
	src/java/org/apache/cassandra/service/StorageService.java
	src/java/org/apache/cassandra/service/StorageServiceMBean.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/de1a96c8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/de1a96c8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/de1a96c8

Branch: refs/heads/trunk
Commit: de1a96c8d401f706558e5841dfce5da85b408b26
Parents: 848352f a4e1182
Author: Tyler Hobbs <tylerlhobbs@gmail.com>
Authored: Thu Apr 21 16:18:06 2016 -0500
Committer: Tyler Hobbs <tylerlhobbs@gmail.com>
Committed: Thu Apr 21 16:18:06 2016 -0500

----------------------------------------------------------------------
 CHANGES.txt                                     |  2 ++
 .../org/apache/cassandra/config/Schema.java     | 13 ++++++++++
 src/java/org/apache/cassandra/db/Keyspace.java  |  5 ++++
 .../cassandra/db/SizeEstimatesRecorder.java     |  2 +-
 .../org/apache/cassandra/dht/BootStrapper.java  |  3 +--
 .../service/PendingRangeCalculatorService.java  |  6 +++--
 .../cassandra/service/StorageService.java       | 27 +++++++++++---------
 .../cassandra/service/StorageServiceMBean.java  |  2 ++
 .../org/apache/cassandra/tools/NodeProbe.java   |  5 ++++
 .../org/apache/cassandra/tools/NodeTool.java    | 21 ++++++++++++---
 .../cassandra/tools/nodetool/Cleanup.java       |  2 +-
 .../apache/cassandra/tools/nodetool/Repair.java |  2 +-
 .../apache/cassandra/dht/BootStrapperTest.java  |  2 +-
 .../cassandra/locator/SimpleStrategyTest.java   |  4 +--
 .../service/LeaveAndBootstrapTest.java          |  4 +--
 .../org/apache/cassandra/service/MoveTest.java  |  4 +--
 16 files changed, 75 insertions(+), 29 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cassandra/blob/de1a96c8/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index a0c7df6,0ec6aef..0704c57
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,61 -1,6 +1,63 @@@
 -3.0.6
 +3.6
 + * Support large partitions on the 3.0 sstable format (CASSANDRA-11206)
 + * JSON datetime formatting needs timezone (CASSANDRA-11137)
 + * Add support to rebuild from specific range (CASSANDRA-10409)
 + * Optimize the overlapping lookup by calculating all the
 +   bounds in advance (CASSANDRA-11571)
 + * Support json/yaml output in noetool tablestats (CASSANDRA-5977)
 + * (stress) Add datacenter option to -node options (CASSANDRA-11591)
 + * Fix handling of empty slices (CASSANDRA-11513)
 + * Make number of cores used by cqlsh COPY visible to testing code (CASSANDRA-11437)
 + * Allow filtering on clustering columns for queries without secondary indexes (CASSANDRA-11310)
 + * Refactor Restriction hierarchy (CASSANDRA-11354)
 + * Eliminate allocations in R/W path (CASSANDRA-11421)
 + * Update Netty to 4.0.36 (CASSANDRA-11567)
 + * Fix PER PARTITION LIMIT for queries requiring post-query ordering (CASSANDRA-11556)
 + * Allow instantiation of UDTs and tuples in UDFs (CASSANDRA-10818)
 + * Support UDT in CQLSSTableWriter (CASSANDRA-10624)
 + * Support for non-frozen user-defined types, updating
 +   individual fields of user-defined types (CASSANDRA-7423)
 + * Make LZ4 compression level configurable (CASSANDRA-11051)
 + * Allow per-partition LIMIT clause in CQL (CASSANDRA-7017)
 + * Make custom filtering more extensible with UserExpression (CASSANDRA-11295)
 + * Improve field-checking and error reporting in cassandra.yaml (CASSANDRA-10649)
 + * Print CAS stats in nodetool proxyhistograms (CASSANDRA-11507)
 + * More user friendly error when providing an invalid token to nodetool (CASSANDRA-9348)
 + * Add static column support to SASI index (CASSANDRA-11183)
 + * Support EQ/PREFIX queries in SASI CONTAINS mode without tokenization (CASSANDRA-11434)
 + * Support LIKE operator in prepared statements (CASSANDRA-11456)
 + * Add a command to see if a Materialized View has finished building (CASSANDRA-9967)
 + * Log endpoint and port associated with streaming operation (CASSANDRA-8777)
 + * Print sensible units for all log messages (CASSANDRA-9692)
 + * Upgrade Netty to version 4.0.34 (CASSANDRA-11096)
 + * Break the CQL grammar into separate Parser and Lexer (CASSANDRA-11372)
 + * Compress only inter-dc traffic by default (CASSANDRA-8888)
 + * Add metrics to track write amplification (CASSANDRA-11420)
 + * cassandra-stress: cannot handle "value-less" tables (CASSANDRA-7739)
 + * Add/drop multiple columns in one ALTER TABLE statement (CASSANDRA-10411)
 + * Add require_endpoint_verification opt for internode encryption (CASSANDRA-9220)
 + * Add auto import java.util for UDF code block (CASSANDRA-11392)
 + * Add --hex-format option to nodetool getsstables (CASSANDRA-11337)
 + * sstablemetadata should print sstable min/max token (CASSANDRA-7159)
 + * Do not wrap CassandraException in TriggerExecutor (CASSANDRA-9421)
 + * COPY TO should have higher double precision (CASSANDRA-11255)
 + * Stress should exit with non-zero status after failure (CASSANDRA-10340)
 + * Add client to cqlsh SHOW_SESSION (CASSANDRA-8958)
 + * Fix nodetool tablestats keyspace level metrics (CASSANDRA-11226)
 + * Store repair options in parent_repair_history (CASSANDRA-11244)
 + * Print current leveling in sstableofflinerelevel (CASSANDRA-9588)
 + * Change repair message for keyspaces with RF 1 (CASSANDRA-11203)
 + * Remove hard-coded SSL cipher suites and protocols (CASSANDRA-10508)
 + * Improve concurrency in CompactionStrategyManager (CASSANDRA-10099)
 + * (cqlsh) interpret CQL type for formatting blobs (CASSANDRA-11274)
 + * Refuse to start and print txn log information in case of disk
 +   corruption (CASSANDRA-10112)
 + * Resolve some eclipse-warnings (CASSANDRA-11086)
 + * (cqlsh) Show static columns in a different color (CASSANDRA-11059)
 + * Allow to remove TTLs on table with default_time_to_live (CASSANDRA-11207)
 +Merged from 3.0:
+  * Ignore all LocalStrategy keyspaces for streaming and other related
+    operations (CASSANDRA-11627)
   * Ensure columnfilter covers indexed columns for thrift 2i queries (CASSANDRA-11523)
   * Only open one sstable scanner per sstable (CASSANDRA-11412)
   * Option to specify ProtocolVersion in cassandra-stress (CASSANDRA-11410)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/de1a96c8/src/java/org/apache/cassandra/db/Keyspace.java
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/de1a96c8/src/java/org/apache/cassandra/dht/BootStrapper.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/dht/BootStrapper.java
index d10aa3b,c44bad7..902c584
--- a/src/java/org/apache/cassandra/dht/BootStrapper.java
+++ b/src/java/org/apache/cassandra/dht/BootStrapper.java
@@@ -74,11 -73,10 +73,11 @@@ public class BootStrapper extends Progr
                                                     "Bootstrap",
                                                     useStrictConsistency,
                                                     DatabaseDescriptor.getEndpointSnitch(),
 -                                                   stateStore);
 +                                                   stateStore,
 +                                                   true);
          streamer.addSourceFilter(new RangeStreamer.FailureDetectorSourceFilter(FailureDetector.instance));
  
-         for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())
+         for (String keyspaceName : Schema.instance.getNonLocalStrategyKeyspaces())
          {
              AbstractReplicationStrategy strategy = Keyspace.open(keyspaceName).getReplicationStrategy();
              streamer.addRanges(keyspaceName, strategy.getPendingAddressRanges(tokenMetadata,
tokens, address));

http://git-wip-us.apache.org/repos/asf/cassandra/blob/de1a96c8/src/java/org/apache/cassandra/service/StorageService.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index 6051567,6fa130a..56d9b4a
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -1164,35 -1160,8 +1164,35 @@@ public class StorageService extends Not
              if (sourceDc != null)
                  streamer.addSourceFilter(new RangeStreamer.SingleDatacenterFilter(DatabaseDescriptor.getEndpointSnitch(),
sourceDc));
  
 -            for (String keyspaceName : Schema.instance.getNonLocalStrategyKeyspaces())
 -                streamer.addRanges(keyspaceName, getLocalRanges(keyspaceName));
 +            if (keyspace == null)
 +            {
-                 for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())
++                for (String keyspaceName : Schema.instance.getNonLocalStrategyKeyspaces())
 +                    streamer.addRanges(keyspaceName, getLocalRanges(keyspaceName));
 +            }
 +            else if (tokens == null)
 +            {
 +                streamer.addRanges(keyspace, getLocalRanges(keyspace));
 +            }
 +            else
 +            {
 +                Token.TokenFactory factory = getTokenFactory();
 +                List<Range<Token>> ranges = new ArrayList<>();
 +                Pattern rangePattern = Pattern.compile("\\(\\s*(-?\\w+)\\s*,\\s*(-?\\w+)\\s*\\]");
 +                try (Scanner tokenScanner = new Scanner(tokens))
 +                {
 +                    while (tokenScanner.findInLine(rangePattern) != null)
 +                    {
 +                        MatchResult range = tokenScanner.match();
 +                        Token startToken = factory.fromString(range.group(1));
 +                        Token endToken = factory.fromString(range.group(2));
 +                        logger.info(String.format("adding range: (%s,%s]", startToken, endToken));
 +                        ranges.add(new Range<>(startToken, endToken));
 +                    }
 +                    if (tokenScanner.hasNext())
 +                        throw new IllegalArgumentException("Unexpected string: " + tokenScanner.next());
 +                }
 +                streamer.addRanges(keyspace, ranges);
 +            }
  
              StreamResultFuture resultFuture = streamer.fetchAsync();
              // wait for result
@@@ -4119,10 -3944,8 +4119,10 @@@
          if (!replicatingNodes.isEmpty())
              throw new UnsupportedOperationException("This node is already processing a removal.
Wait for it to complete, or use 'removenode force' if this has failed.");
  
 +        Collection<Token> tokens = tokenMetadata.getTokens(endpoint);
 +
          // Find the endpoints that are going to become responsible for data
-         for (String keyspaceName : Schema.instance.getNonSystemKeyspaces())
+         for (String keyspaceName : Schema.instance.getNonLocalStrategyKeyspaces())
          {
              // if the replication factor is 1 the data is lost so we shouldn't wait for
confirmation
              if (Keyspace.open(keyspaceName).getReplicationStrategy().getReplicationFactor()
== 1)
@@@ -4434,32 -4257,16 +4434,35 @@@
          return Collections.unmodifiableList(keyspaceNamesList);
      }
  
- 
      public List<String> getNonSystemKeyspaces()
      {
-         List<String> keyspaceNamesList = new ArrayList<>(Schema.instance.getNonSystemKeyspaces());
-         return Collections.unmodifiableList(keyspaceNamesList);
+         return Collections.unmodifiableList(Schema.instance.getNonSystemKeyspaces());
+     }
+ 
+     public List<String> getNonLocalStrategyKeyspaces()
+     {
+         return Collections.unmodifiableList(Schema.instance.getNonLocalStrategyKeyspaces());
      }
  
 +    public Map<String, String> getViewBuildStatuses(String keyspace, String view)
 +    {
 +        Map<UUID, String> coreViewStatus = SystemDistributedKeyspace.viewStatus(keyspace,
view);
 +        Map<InetAddress, UUID> hostIdToEndpoint = tokenMetadata.getEndpointToHostIdMapForReading();
 +        Map<String, String> result = new HashMap<>();
 +
 +        for (Map.Entry<InetAddress, UUID> entry : hostIdToEndpoint.entrySet())
 +        {
 +            UUID hostId = entry.getValue();
 +            InetAddress endpoint = entry.getKey();
 +            result.put(endpoint.toString(),
 +                       coreViewStatus.containsKey(hostId)
 +                       ? coreViewStatus.get(hostId)
 +                       : "UNKNOWN");
 +        }
 +
 +        return Collections.unmodifiableMap(result);
 +    }
 +
      public void updateSnitch(String epSnitchClassName, Boolean dynamic, Integer dynamicUpdateInterval,
Integer dynamicResetInterval, Double dynamicBadnessThreshold) throws ClassNotFoundException
      {
          IEndpointSnitch oldSnitch = DatabaseDescriptor.getEndpointSnitch();

http://git-wip-us.apache.org/repos/asf/cassandra/blob/de1a96c8/src/java/org/apache/cassandra/service/StorageServiceMBean.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/service/StorageServiceMBean.java
index f48e469,e111bc4..8978472
--- a/src/java/org/apache/cassandra/service/StorageServiceMBean.java
+++ b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
@@@ -452,8 -446,8 +452,10 @@@ public interface StorageServiceMBean ex
  
      public List<String> getNonSystemKeyspaces();
  
+     public List<String> getNonLocalStrategyKeyspaces();
+ 
 +    public Map<String, String> getViewBuildStatuses(String keyspace, String view);
 +
      /**
       * Change endpointsnitch class and dynamic-ness (and dynamic attributes) at runtime
       * @param epSnitchClassName        the canonical path name for a class implementing
IEndpointSnitch

http://git-wip-us.apache.org/repos/asf/cassandra/blob/de1a96c8/src/java/org/apache/cassandra/tools/NodeProbe.java
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/de1a96c8/src/java/org/apache/cassandra/tools/NodeTool.java
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/de1a96c8/test/unit/org/apache/cassandra/dht/BootStrapperTest.java
----------------------------------------------------------------------


Mime
View raw message