cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.X
Date Thu, 05 Jan 2017 21:30:45 GMT
Merge branch 'cassandra-3.0' into cassandra-3.X


Branch: refs/heads/trunk
Commit: eca51a0348141f45ea9bb31b76571ed81284c32c
Parents: 5f3b359 6e716c6
Author: Robert Stupp <>
Authored: Thu Jan 5 22:26:48 2017 +0100
Committer: Robert Stupp <>
Committed: Thu Jan 5 22:26:48 2017 +0100

 CHANGES.txt                                     |  2 +
 NEWS.txt                                        |  3 ++
 doc/cql3/CQL.textile                            |  2 +-
 lib/jsr223/clojure/README.txt                   |  8 ---
 lib/jsr223/groovy/README.txt                    | 35 -------------
 lib/jsr223/jaskell/README.txt                   |  5 --
 lib/jsr223/jruby/README.txt                     | 54 --------------------
 lib/jsr223/jython/README.txt                    | 33 ------------
 lib/jsr223/scala/README.txt                     | 37 --------------
 .../cql3/functions/   | 35 +++++--------
 .../cql3/validation/entities/  |  3 +-
 .../validation/entities/     | 12 ++++-
 12 files changed, 31 insertions(+), 198 deletions(-)
diff --cc CHANGES.txt
index cacb7cd,77a310d..66c2ee6
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,136 -1,5 +1,138 @@@
 - * Fixed flacky SSTableRewriterTest: check file counts before calling validateCFS (CASSANDRA-12348)
 + * Use new token allocation for non bootstrap case as well (CASSANDRA-13080)
 + * Avoid byte-array copy when key cache is disabled (CASSANDRA-13084)
 + * More fixes to the TokenAllocator (CASSANDRA-12990)
 + * Require forceful decommission if number of nodes is less than replication factor (CASSANDRA-12510)
 + * Allow IN restrictions on column families with collections (CASSANDRA-12654)
 + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
 + * nodetool stopdaemon errors out (CASSANDRA-13030)
 + * Log message size in trace message in OutboundTcpConnection (CASSANDRA-13028)
 + * Add timeUnit Days for cassandra-stress (CASSANDRA-13029)
 + * Add mutation size and batch metrics (CASSANDRA-12649)
 + * Add method to get size of endpoints to TokenMetadata (CASSANDRA-12999)
 + * Fix primary index calculation for SASI (CASSANDRA-12910)
 + * Expose time spent waiting in thread pool queue (CASSANDRA-8398)
 + * Conditionally update index built status to avoid unnecessary flushes (CASSANDRA-12969)
 + * NoReplicationTokenAllocator should work with zero replication factor (CASSANDRA-12983)
 + * cqlsh auto completion: refactor definition of compaction strategy options (CASSANDRA-12946)
 + * Add support for arithmetic operators (CASSANDRA-11935)
 + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
++Merged from 3.0:
++ * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
 + * Fixed query monitoring for range queries (CASSANDRA-13050)
 + * Remove outboundBindAny configuration property (CASSANDRA-12673)
 + * Use correct bounds for all-data range when filtering (CASSANDRA-12666)
 + * Remove timing window in test case (CASSANDRA-12875)
 + * Resolve unit testing without JCE security libraries installed (CASSANDRA-12945)
 + * Fix inconsistencies in cassandra-stress load balancing policy (CASSANDRA-12919)
 + * Fix validation of non-frozen UDT cells (CASSANDRA-12916)
 + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903)
 + * Fix Murmur3PartitionerTest (CASSANDRA-12858)
 + * Move cqlsh syntax rules into separate module and allow easier customization (CASSANDRA-12897)
 + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283)
 + * Fix cassandra-stress truncate option (CASSANDRA-12695)
 + * Fix crossNode value when receiving messages (CASSANDRA-12791)
 + * Don't load MX4J beans twice (CASSANDRA-12869)
 + * Extend native protocol request flags, add versions to SUPPORTED, and introduce ProtocolVersion
enum (CASSANDRA-12838)
 + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836)
 + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845)
 + * Properly format IPv6 addresses when logging JMX service URL (CASSANDRA-12454)
 + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777)
 + * Use non-token restrictions for bounds when token restrictions are overridden (CASSANDRA-12419)
 + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803)
 + * Use different build directories for Eclipse and Ant (CASSANDRA-12466)
 + * Avoid potential AttributeError in cqlsh due to no table metadata (CASSANDRA-12815)
 + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster (CASSANDRA-12812)
 + * Upgrade commons-codec to 1.9 (CASSANDRA-12790)
 + * Make the fanout size for LeveledCompactionStrategy to be configurable (CASSANDRA-11550)
 + * Add duration data type (CASSANDRA-11873)
 + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784)
 + * Improve sum aggregate functions (CASSANDRA-12417)
 + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in CASSANDRA-10876
 + * cqlsh fails to format collections when using aliases (CASSANDRA-11534)
 + * Check for hash conflicts in prepared statements (CASSANDRA-12733)
 + * Exit query parsing upon first error (CASSANDRA-12598)
 + * Fix cassandra-stress to use single seed in UUID generation (CASSANDRA-12729)
 + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450)
 + * Config class uses boxed types but DD exposes primitive types (CASSANDRA-12199)
 + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461)
 + * Add hint delivery metrics (CASSANDRA-12693)
 + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731)
 + * ColumnIndex does not reuse buffer (CASSANDRA-12502)
 + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697)
 + * Upgrade metrics-reporter dependencies (CASSANDRA-12089)
 + * Tune compaction thread count via nodetool (CASSANDRA-12248)
 + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232)
 + * Include repair session IDs in repair start message (CASSANDRA-12532)
 + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039)
 + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667)
 + * Support optional backpressure strategies at the coordinator (CASSANDRA-9318)
 + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647)
 + * Fix cassandra-stress graphing (CASSANDRA-12237)
 + * Allow filtering on partition key columns for queries without secondary indexes (CASSANDRA-11031)
 + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585)
 + * Add JMH benchmarks.jar (CASSANDRA-12586)
 + * Cleanup uses of AlterTableStatementColumn (CASSANDRA-12567)
 + * Add keep-alive to streaming (CASSANDRA-11841)
 + * Tracing payload is passed through newSession(..) (CASSANDRA-11706)
 + * avoid deleting non existing sstable files and improve related log messages (CASSANDRA-12261)
 + * json/yaml output format for nodetool compactionhistory (CASSANDRA-12486)
 + * Retry all internode messages once after a connection is
 +   closed and reopened (CASSANDRA-12192)
 + * Add support to rebuild from targeted replica (CASSANDRA-9875)
 + * Add sequence distribution type to cassandra stress (CASSANDRA-12490)
 + * "SELECT * FROM foo LIMIT ;" does not error out (CASSANDRA-12154)
 + * Define executeLocally() at the ReadQuery Level (CASSANDRA-12474)
 + * Extend read/write failure messages with a map of replica addresses
 +   to error codes in the v5 native protocol (CASSANDRA-12311)
 + * Fix rebuild of SASI indexes with existing index files (CASSANDRA-12374)
 + * Let DatabaseDescriptor not implicitly startup services (CASSANDRA-9054, 12550)
 + * Fix clustering indexes in presence of static columns in SASI (CASSANDRA-12378)
 + * Fix queries on columns with reversed type on SASI indexes (CASSANDRA-12223)
 + * Added slow query log (CASSANDRA-12403)
 + * Count full coordinated request against timeout (CASSANDRA-12256)
 + * Allow TTL with null value on insert and update (CASSANDRA-12216)
 + * Make decommission operation resumable (CASSANDRA-12008)
 + * Add support to one-way targeted repair (CASSANDRA-9876)
 + * Remove clientutil jar (CASSANDRA-11635)
 + * Fix compaction throughput throttle (CASSANDRA-12366, CASSANDRA-12717)
 + * Delay releasing Memtable memory on flush until PostFlush has finished running (CASSANDRA-12358)
 + * Cassandra stress should dump all setting on startup (CASSANDRA-11914)
 + * Make it possible to compact a given token range (CASSANDRA-10643)
 + * Allow updating DynamicEndpointSnitch properties via JMX (CASSANDRA-12179)
 + * Collect metrics on queries by consistency level (CASSANDRA-7384)
 + * Add support for GROUP BY to SELECT statement (CASSANDRA-10707)
 + * Deprecate memtable_cleanup_threshold and update default for memtable_flush_writers (CASSANDRA-12228)
 + * Upgrade to OHC 0.4.4 (CASSANDRA-12133)
 + * Add version command to cassandra-stress (CASSANDRA-12258)
 + * Create compaction-stress tool (CASSANDRA-11844)
 + * Garbage-collecting compaction operation and schema option (CASSANDRA-7019)
 + * Add beta protocol flag for v5 native protocol (CASSANDRA-12142)
 + * Support filtering on non-PRIMARY KEY columns in the CREATE
 +   MATERIALIZED VIEW statement's WHERE clause (CASSANDRA-10368)
 + * Unify STDOUT and SYSTEMLOG logback format (CASSANDRA-12004)
 + * COPY FROM should raise error for non-existing input files (CASSANDRA-12174)
 + * Faster write path (CASSANDRA-12269)
 + * Option to leave omitted columns in INSERT JSON unset (CASSANDRA-11424)
 + * Support json/yaml output in nodetool tpstats (CASSANDRA-12035)
 + * Expose metrics for successful/failed authentication attempts (CASSANDRA-10635)
 + * Prepend snapshot name with "truncated" or "dropped" when a snapshot
 +   is taken before truncating or dropping a table (CASSANDRA-12178)
 + * Optimize RestrictionSet (CASSANDRA-12153)
 + * cqlsh does not automatically downgrade CQL version (CASSANDRA-12150)
 + * Omit (de)serialization of state variable in UDAs (CASSANDRA-9613)
 + * Create a system table to expose prepared statements (CASSANDRA-8831)
 + * Reuse DataOutputBuffer from ColumnIndex (CASSANDRA-11970)
 + * Remove DatabaseDescriptor dependency from SegmentedFile (CASSANDRA-11580)
 + * Add supplied username to authentication error messages (CASSANDRA-12076)
 + * Remove pre-startup check for open JMX port (CASSANDRA-12074)
 + * Remove compaction Severity from DynamicEndpointSnitch (CASSANDRA-11738)
 + * Restore resumable hints delivery (CASSANDRA-11960)
 + * Properly report LWT contention (CASSANDRA-12626)
 +Merged from 3.0:
   * Fix deserialization of 2.x DeletedCells (CASSANDRA-12620)
   * Add parent repair session id to anticompaction log message (CASSANDRA-12186)
   * Improve contention handling on failure to acquire MV lock for streaming and hints (CASSANDRA-12905)
diff --cc NEWS.txt
index e6c8c10,b4e6551..9f376ff
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -21,125 -21,23 +21,128 @@@ Upgradin
     - Specifying the default_time_to_live option when creating or altering a
       materialized view was erroneously accepted (and ignored). It is now
       properly rejected.
+    - Only Java and JavaScript are now supported UDF languages.
+      The sandbox in 3.0 already prevented the use of script languages except Java
+      and JavaScript.
 -   - memtable_allocation_type: offheap_buffers is no longer allowed to be specified in the
3.0 series.
 -     This was an oversight that can cause segfaults. Offheap was re-introduced in 3.4 see
 -     and CASSANDRA-9472 for details.
 +New features
 +   - New `DurationType` (cql duration). See CASSANDRA-11873
 +   - Runtime modification of concurrent_compactors is now available via nodetool
 +   - Support for the assignment operators +=/-= has been added for update queries.
 +   - An Index implementation may now provide a task which runs prior to joining
 +     the ring. See CASSANDRA-12039
 +   - Filtering on partition key columns is now also supported for queries without
 +     secondary indexes.
 +   - A slow query log has been added: slow queries will be logged at DEBUG level.
 +     For more details refer to CASSANDRA-12403 and slow_query_log_timeout_in_ms
 +     in cassandra.yaml.
 +   - Support for GROUP BY queries has been added.
 +   - A new compaction-stress tool has been added to test the throughput of compaction
 +     for any cassandra-stress user schema.  see compaction-stress help for how to use.
 +   - Compaction can now take into account overlapping tables that don't take part
 +     in the compaction to look for deleted or overwritten data in the compacted tables.
 +     Then such data is found, it can be safely discarded, which in turn should enable
 +     the removal of tombstones over that data.
 +     The behavior can be engaged in two ways:
 +       - as a "nodetool garbagecollect -g CELL/ROW" operation, which applies
 +         single-table compaction on all sstables to discard deleted data in one step.
 +       - as a "provide_overlapping_tombstones:CELL/ROW/NONE" compaction strategy flag,
 +         which uses overlapping tables as a source of deletions/overwrites during all
 +         compactions.
 +     The argument specifies the granularity at which deleted data is to be found:
 +       - If ROW is specified, only whole deleted rows (or sets of rows) will be
 +         discarded.
 +       - If CELL is specified, any columns whose value is overwritten or deleted
 +         will also be discarded.
 +       - NONE (default) specifies the old behavior, overlapping tables are not used to
 +         decide when to discard data.
 +     Which option to use depends on your workload, both ROW and CELL increase the
 +     disk load on compaction (especially with the size-tiered compaction strategy),
 +     with CELL being more resource-intensive. Both should lead to better read
 +     performance if deleting rows (resp. overwriting or deleting cells) is common.
 +   - Prepared statements are now persisted in the table prepared_statements in
 +     the system keyspace. Upon startup, this table is used to preload all
 +     previously prepared statements - i.e. in many cases clients do not need to
 +     re-prepare statements against restarted nodes.
 +   - cqlsh can now connect to older Cassandra versions by downgrading the native
 +     protocol version. Please note that this is currently not part of our release
 +     testing and, as a consequence, it is not guaranteed to work in all cases.
 +     See CASSANDRA-12150 for more details.
 +   - Snapshots that are automatically taken before a table is dropped or truncated
 +     will have a "dropped" or "truncated" prefix on their snapshot tag name.
 +   - Metrics are exposed for successful and failed authentication attempts.
 +     These can be located using the object names org.apache.cassandra.metrics:type=Client,name=AuthSuccess
 +     and org.apache.cassandra.metrics:type=Client,name=AuthFailure respectively.
 +   - Add support to "unset" JSON fields in prepared statements by specifying DEFAULT UNSET.
 +     See CASSANDRA-11424 for details
 +   - Allow TTL with null value on insert and update. It will be treated as equivalent to
inserting a 0.
 +   - Removed outboundBindAny configuration property. See CASSANDRA-12673 for details.
 +    - Request timeouts in cassandra.yaml (read_request_timeout_in_ms, etc) now apply to
 +      "full" request time on the coordinator.  Previously, they only covered the time from
 +      when the coordinator sent a message to a replica until the time that the replica
 +      responded.  Additionally, the previous behavior was to reset the timeout when performing
 +      a read repair, making a second read to fix a short read, and when subranges were read
 +      as part of a range scan or secondary index query.  In 3.10 and higher, the timeout
 +      is no longer reset for these "subqueries".  The entire request must complete within
 +      the specified timeout.  As a consequence, your timeouts may need to be adjusted
 +      to account for this.  See CASSANDRA-12256 for more details.
 +    - Logs written to stdout are now consistent with logs written to files.
 +      Time is now local (it was UTC on the console and local in files). Date, thread, file
 +      and line info where added to stdout. (see CASSANDRA-12004)
 +    - The 'clientutil' jar, which has been somewhat broken on the 3.x branch, is not longer
 +      The features provided by that jar are provided by any good java driver and we advise
relying on drivers rather on
 +      that jar, but if you need that jar for backward compatiblity until you do so, you
should use the version provided
 +      on previous Cassandra branch, like the 3.0 branch (by design, the functionality provided
by that jar are stable
 +      accross versions so using the 3.0 jar for a client connecting to 3.x should work without
 +    - (Tools development) DatabaseDescriptor no longer implicitly startups components/services
 +      commit log replay. This may break existing 3rd party tools and clients. In order to
 +      a standalone tool or client application, use the DatabaseDescriptor.toolInitialization()
 +      DatabaseDescriptor.clientInitialization() methods. Tool initialization sets up partitioner,
 +      snitch, encryption context. Client initialization just applies the configuration but
does not
 +      setup anything. Instead of using Config.setClientMode() or Config.isClientMode(),
which are
 +      deprecated now, use one of the appropiate new methods in DatabaseDescriptor.
 +    - Application layer keep-alives were added to the streaming protocol to prevent idle
incoming connections from
 +      timing out and failing the stream session (CASSANDRA-11839). This effectively deprecates
the streaming_socket_timeout_in_ms
 +      property in favor of streaming_keep_alive_period_in_secs. See cassandra.yaml for more
details about this property.
 +    - Duration litterals support the ISO 8601 format. By consequence, identifiers matching
that format
 +      (e.g P2Y or P1MT6H) will not be supported anymore (CASSANDRA-11873).
 +New features
 +   - Shared pool threads are now named according to the stage they are executing
 +     tasks for. Thread names mentioned in traced queries change accordingly.
 +   - A new option has been added to cassandra-stress "-rate fixed={number}/s"
 +     that forces a scheduled rate of operations/sec over time. Using this, stress can
 +     accurately account for coordinated ommission from the stress process.
 +   - The cassandra-stress "-rate limit=" option has been renamed to "-rate throttle="
 +   - hdr histograms have been added to stress runs, it's output can be saved to disk using:
 +     "-log hdrfile=" option. This histogram includes response/service/wait times when used
with the
 +     fixed or throttle rate options.  The histogram file can be plotted on
 +   - TimeWindowCompactionStrategy has been added. This has proven to be a better approach
 +     to time series compaction and new tables should use this instead of DTCS. See
 +     CASSANDRA-9666 for details.
 +   - Change-Data-Capture is now available. See cassandra.yaml and for cdc-specific flags
 +     a brief explanation of on-disk locations for archived data in CommitLog form. This
 +     be enabled via ALTER TABLE ... WITH cdc=true.
 +     Upon flush, CommitLogSegments containing data for CDC-enabled tables are moved to
 +     the data/cdc_raw directory until removed by the user and writes to CDC-enabled tables
 +     will be rejected with a WriteTimeoutException once cdc_total_space_in_mb is reached
 +     between unflushed CommitLogSegments and cdc_raw.
 +     NOTE: CDC is disabled by default in the .yaml file. Do not enable CDC on a mixed-version
 +     cluster as it will lead to exceptions which can interrupt traffic. Once all nodes
 +     have been upgraded to 3.8 it is safe to enable this feature and restart the cluster.
diff --cc src/java/org/apache/cassandra/cql3/functions/
index 3ad60d0,47deafa..c568972
--- a/src/java/org/apache/cassandra/cql3/functions/
+++ b/src/java/org/apache/cassandra/cql3/functions/
@@@ -27,7 -27,9 +27,10 @@@ import java.util.*
  import java.util.concurrent.ExecutorService;
  import javax.script.*;
 +import jdk.nashorn.api.scripting.AbstractJSObject;
+ import jdk.nashorn.api.scripting.ClassFilter;
+ import jdk.nashorn.api.scripting.NashornScriptEngine;
+ import jdk.nashorn.api.scripting.NashornScriptEngineFactory;
  import org.apache.cassandra.concurrent.NamedThreadFactory;
  import org.apache.cassandra.cql3.ColumnIdentifier;
  import org.apache.cassandra.db.marshal.AbstractType;
@@@ -158,13 -149,6 +152,10 @@@ final class ScriptBasedUDFunction exten
              throw new InvalidRequestException(
                                               String.format("Failed to compile function '%s'
for language %s: %s", name, language, e));
 +        // It's not always possible to simply pass a plain Java object as a binding to Nashorn
 +        // let the script execute methods on it.
-         udfContextBinding =
-             ("Oracle Nashorn".equals(((ScriptEngine) scriptEngine).getFactory().getEngineName()))
-                 ? new UDFContextWrapper()
-                 : udfContext;
++        udfContextBinding = new UDFContextWrapper();
      protected ExecutorService executor()

View raw message