cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tylerho...@apache.org
Subject [2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Date Tue, 14 Oct 2014 18:23:39 GMT
Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
	CHANGES.txt
	src/java/org/apache/cassandra/transport/CBUtil.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3a94b337
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3a94b337
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3a94b337

Branch: refs/heads/trunk
Commit: 3a94b3379ec9efb012d227af684c61ca19291b99
Parents: e57d027 781018c
Author: Tyler Hobbs <tylerhobbs@apache.org>
Authored: Tue Oct 14 13:22:42 2014 -0500
Committer: Tyler Hobbs <tylerhobbs@apache.org>
Committed: Tue Oct 14 13:22:42 2014 -0500

----------------------------------------------------------------------
 CHANGES.txt                                     |  1 +
 .../apache/cassandra/db/marshal/AsciiType.java  | 27 ++++++++++-
 .../org/apache/cassandra/transport/CBUtil.java  | 51 +++++++++++++++++---
 3 files changed, 71 insertions(+), 8 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3a94b337/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index 74647c8,432750a..47bea3a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,76 -1,5 +1,77 @@@
 -2.0.11:
 +2.1.1
 + * Fix high memory use due to tracking reads on incrementally opened sstable readers (CASSANDRA-8066)
 + * Fix EXECUTE request with skipMetadata=false returning no metadata
 +   (CASSANDRA-8054)
 + * Allow concurrent use of CQLBulkOutputFormat (CASSANDRA-7776)
 + * Shutdown JVM on OOM (CASSANDRA-7507)
 + * Upgrade netty version and enable epoll event loop (CASSANDRA-7761)
 + * Don't duplicate sstables smaller than split size when using
 +   the sstablesplitter tool (CASSANDRA-7616)
 + * Avoid re-parsing already prepared statements (CASSANDRA-7923)
 + * Fix some Thrift slice deletions and updates of COMPACT STORAGE
 +   tables with some clustering columns omitted (CASSANDRA-7990)
 + * Fix filtering for CONTAINS on sets (CASSANDRA-8033)
 + * Properly track added size (CASSANDRA-7239)
 + * Allow compilation in java 8 (CASSANDRA-7208)
 + * Fix Assertion error on RangeTombstoneList diff (CASSANDRA-8013)
 + * Release references to overlapping sstables during compaction (CASSANDRA-7819)
 + * Send notification when opening compaction results early (CASSANDRA-8034)
 + * Make native server start block until properly bound (CASSANDRA-7885)
 + * (cqlsh) Fix IPv6 support (CASSANDRA-7988)
 + * Ignore fat clients when checking for endpoint collision (CASSANDRA-7939)
 + * Make sstablerepairedset take a list of files (CASSANDRA-7995)
 + * (cqlsh) Tab completeion for indexes on map keys (CASSANDRA-7972)
 + * (cqlsh) Fix UDT field selection in select clause (CASSANDRA-7891)
 + * Fix resource leak in event of corrupt sstable
 + * (cqlsh) Add command line option for cqlshrc file path (CASSANDRA-7131)
 + * Provide visibility into prepared statements churn (CASSANDRA-7921, CASSANDRA-7930)
 + * Invalidate prepared statements when their keyspace or table is
 +   dropped (CASSANDRA-7566)
 + * cassandra-stress: fix support for NetworkTopologyStrategy (CASSANDRA-7945)
 + * Fix saving caches when a table is dropped (CASSANDRA-7784)
 + * Add better error checking of new stress profile (CASSANDRA-7716)
 + * Use ThreadLocalRandom and remove FBUtilities.threadLocalRandom (CASSANDRA-7934)
 + * Prevent operator mistakes due to simultaneous bootstrap (CASSANDRA-7069)
 + * cassandra-stress supports whitelist mode for node config (CASSANDRA-7658)
 + * GCInspector more closely tracks GC; cassandra-stress and nodetool report it (CASSANDRA-7916)
 + * nodetool won't output bogus ownership info without a keyspace (CASSANDRA-7173)
 + * Add human readable option to nodetool commands (CASSANDRA-5433)
 + * Don't try to set repairedAt on old sstables (CASSANDRA-7913)
 + * Add metrics for tracking PreparedStatement use (CASSANDRA-7719)
 + * (cqlsh) tab-completion for triggers (CASSANDRA-7824)
 + * (cqlsh) Support for query paging (CASSANDRA-7514)
 + * (cqlsh) Show progress of COPY operations (CASSANDRA-7789)
 + * Add syntax to remove multiple elements from a map (CASSANDRA-6599)
 + * Support non-equals conditions in lightweight transactions (CASSANDRA-6839)
 + * Add IF [NOT] EXISTS to create/drop triggers (CASSANDRA-7606)
 + * (cqlsh) Display the current logged-in user (CASSANDRA-7785)
 + * (cqlsh) Don't ignore CTRL-C during COPY FROM execution (CASSANDRA-7815)
 + * (cqlsh) Order UDTs according to cross-type dependencies in DESCRIBE
 +   output (CASSANDRA-7659)
 + * (cqlsh) Fix handling of CAS statement results (CASSANDRA-7671)
 + * (cqlsh) COPY TO/FROM improvements (CASSANDRA-7405)
 + * Support list index operations with conditions (CASSANDRA-7499)
 + * Add max live/tombstoned cells to nodetool cfstats output (CASSANDRA-7731)
 + * Validate IPv6 wildcard addresses properly (CASSANDRA-7680)
 + * (cqlsh) Error when tracing query (CASSANDRA-7613)
 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569)
 + * SSTableExport uses correct validator to create string representation of partition
 +   keys (CASSANDRA-7498)
 + * Avoid NPEs when receiving type changes for an unknown keyspace (CASSANDRA-7689)
 + * Add support for custom 2i validation (CASSANDRA-7575)
 + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454)
 + * Add listen_interface and rpc_interface options (CASSANDRA-7417)
 + * Improve schema merge performance (CASSANDRA-7444)
 + * Adjust MT depth based on # of partition validating (CASSANDRA-5263)
 + * Optimise NativeCell comparisons (CASSANDRA-6755)
 + * Configurable client timeout for cqlsh (CASSANDRA-7516)
 + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
 + * Make repair -pr work with -local (CASSANDRA-7450)
 + * Fix error in sstableloader with -cph > 1 (CASSANDRA-8007)
 + * Fix snapshot repair error on indexed tables (CASSANDRA-8020)
 + * Do not exit nodetool repair when receiving JMX NOTIF_LOST (CASSANDRA-7909)
 +Merged from 2.0:
+  * Properly validate ascii and utf8 string literals in CQL queries (CASSANDRA-8101)
   * (cqlsh) Fix autocompletion for alter keyspace (CASSANDRA-8021)
   * Create backup directories for commitlog archiving during startup (CASSANDRA-8111)
   * Reduce totalBlockFor() for LOCAL_* consistency levels (CASSANDRA-8058)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3a94b337/src/java/org/apache/cassandra/db/marshal/AsciiType.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/db/marshal/AsciiType.java
index 3b64cc0,fdccfcd..891a8ed
--- a/src/java/org/apache/cassandra/db/marshal/AsciiType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AsciiType.java
@@@ -18,11 -18,15 +18,16 @@@
  package org.apache.cassandra.db.marshal;
  
  import java.nio.ByteBuffer;
+ import java.nio.CharBuffer;
+ import java.nio.charset.Charset;
+ import java.nio.charset.CharsetEncoder;
+ import java.nio.charset.CharacterCodingException;
  
  import org.apache.cassandra.cql3.CQL3Type;
+ import org.apache.cassandra.serializers.MarshalException;
  import org.apache.cassandra.serializers.TypeSerializer;
  import org.apache.cassandra.serializers.AsciiSerializer;
 +import org.apache.cassandra.utils.ByteBufferUtil;
  
  public class AsciiType extends AbstractType<String>
  {
@@@ -30,9 -34,18 +35,18 @@@
  
      AsciiType() {} // singleton
  
+     private final ThreadLocal<CharsetEncoder> encoder = new ThreadLocal<CharsetEncoder>()
+     {
+         @Override
+         protected CharsetEncoder initialValue()
+         {
+             return Charset.forName("US-ASCII").newEncoder();
+         }
+     };
+ 
      public int compare(ByteBuffer o1, ByteBuffer o2)
      {
 -        return BytesType.bytesCompare(o1, o2);
 +        return ByteBufferUtil.compareUnsigned(o1, o2);
      }
  
      public ByteBuffer fromString(String source)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/3a94b337/src/java/org/apache/cassandra/transport/CBUtil.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/transport/CBUtil.java
index 450dc17,eb32faa..f9425c3
--- a/src/java/org/apache/cassandra/transport/CBUtil.java
+++ b/src/java/org/apache/cassandra/transport/CBUtil.java
@@@ -34,27 -39,38 +38,41 @@@ import io.netty.util.CharsetUtil
  
  import org.apache.cassandra.db.ConsistencyLevel;
  import org.apache.cassandra.db.TypeSizes;
 +import org.apache.cassandra.utils.Pair;
  import org.apache.cassandra.utils.UUIDGen;
+ import org.apache.cassandra.utils.ByteBufferUtil;
  
  /**
 - * ChannelBuffer utility methods.
 + * ByteBuf utility methods.
   * Note that contrarily to ByteBufferUtil, these method do "read" the
 - * ChannelBuffer advancing it's (read) position. They also write by
 + * ByteBuf advancing it's (read) position. They also write by
   * advancing the write position. Functions are also provided to create
 - * ChannelBuffer while avoiding copies.
 + * ByteBuf while avoiding copies.
   */
  public abstract class CBUtil
  {
 +    public static final ByteBufAllocator allocator = new PooledByteBufAllocator(true);
 +
      private CBUtil() {}
  
+     private final static ThreadLocal<CharsetDecoder> decoder = new ThreadLocal<CharsetDecoder>()
+     {
+         @Override
+         protected CharsetDecoder initialValue()
+         {
+             return Charset.forName("UTF-8").newDecoder();
+         }
+     };
+ 
 -    private static String readString(ChannelBuffer cb, int length)
 +    private static String readString(ByteBuf cb, int length)
      {
+         if (length == 0)
+             return "";
+ 
 -        ByteBuffer buffer = cb.toByteBuffer(cb.readerIndex(), length);
++        ByteBuffer buffer = cb.nioBuffer(cb.readerIndex(), length);
          try
          {
-             String str = cb.toString(cb.readerIndex(), length, CharsetUtil.UTF_8);
+             String str = decodeString(buffer);
              cb.readerIndex(cb.readerIndex() + length);
              return str;
          }
@@@ -81,7 -93,30 +95,30 @@@
          }
      }
  
+     // Taken from Netty's ChannelBuffers.decodeString(). We need to use our own decoder
to properly handle invalid
+     // UTF-8 sequences.  See CASSANDRA-8101 for more details.  This can be removed once
https://github.com/netty/netty/pull/2999
+     // is resolved in a release used by Cassandra.
+     private static String decodeString(ByteBuffer src) throws CharacterCodingException
+     {
+         // the decoder needs to be reset every time we use it, hence the copy per thread
+         CharsetDecoder theDecoder = decoder.get();
+         theDecoder.reset();
+ 
+         final CharBuffer dst = CharBuffer.allocate(
+                 (int) ((double) src.remaining() * theDecoder.maxCharsPerByte()));
+ 
+         CoderResult cr = theDecoder.decode(src, dst, true);
+         if (!cr.isUnderflow())
+             cr.throwException();
+ 
+         cr = theDecoder.flush(dst);
+         if (!cr.isUnderflow())
+             cr.throwException();
+ 
+         return dst.flip().toString();
+     }
+ 
 -    public static void writeString(String str, ChannelBuffer cb)
 +    public static void writeString(String str, ByteBuf cb)
      {
          byte[] bytes = str.getBytes(CharsetUtil.UTF_8);
          cb.writeShort(bytes.length);


Mime
View raw message