cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [2/4] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Date Fri, 07 Aug 2015 20:51:24 GMT
Merge branch 'cassandra-2.1' into cassandra-2.2



Branch: refs/heads/trunk
Commit: ce7159e102971c27fd414ce03ced2451c7498bcd
Parents: 8ffeebf 1ecc9cd
Author: Tyler Hobbs <>
Authored: Fri Aug 7 15:49:42 2015 -0500
Committer: Tyler Hobbs <>
Committed: Fri Aug 7 15:49:42 2015 -0500

 NEWS.txt                     | 24 ++++++++++++++++++++++++
 pylib/cqlshlib/ | 28 ++++------------------------
 pylib/cqlshlib/       | 15 +++++++++++++++
 3 files changed, 43 insertions(+), 24 deletions(-)
diff --cc NEWS.txt
index e9c6ef8,0b64e31..1f5e190
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -13,141 -13,35 +13,165 @@@ restore snapshots created with the prev
  'sstableloader' tool. You can upgrade the file format of your snapshots
  using the provided 'sstableupgrade' tool.
 +New features
 +   - Selecting columns,scalar functions, UDT fields, writetime or ttl together
 +     with aggregated is now possible. The value returned for the columns,
 +     scalar functions, UDT fields, writetime and ttl will be the ones for
 +     the first row matching the query.
 +   - Windows is now a supported platform. Powershell execution for startup scripts
 +     is highly recommended and can be enabled via an administrator command-prompt
 +     with: 'powershell set-executionpolicy unrestricted'
 +   - It is now possible to do major compactions when using leveled compaction.
 +     Doing that will take all sstables and compact them out in levels. The
 +     levels will be non overlapping so doing this will still not be something
 +     you want to do very often since it might cause more compactions for a while.
 +     It is also possible to split output when doing a major compaction with
 +     STCS - files will be split in sizes 50%, 25%, 12.5% etc of the total size.
 +     This might be a bit better than old major compactions which created one big
 +     file on disk.
 +   - A new tool has been added bin/sstableverify that checks for errors/bitrot
 +     in all sstables.  Unlike scrub, this is a non-invasive tool.
 +   - Authentication & Authorization APIs have been updated to introduce
 +     roles. Roles and Permissions granted to them are inherited, supporting
 +     role based access control. The role concept supercedes that of users
 +     and CQL constructs such as CREATE USER are deprecated but retained for
 +     compatibility. The requirement to explicitly create Roles in Cassandra
 +     even when auth is handled by an external system has been removed, so
 +     authentication & authorization can be delegated to such systems in their
 +     entirety.
 +   - In addition to the above, Roles are also first class resources and can be the
 +     subject of permissions. Users (roles) can now be granted permissions on other
 +     roles, including CREATE, ALTER, DROP & AUTHORIZE, which removesthe need for
 +     superuser privileges in order to perform user/role management operations.
 +   - Creators of database resources (Keyspaces, Tables, Roles) are now automatically
 +     granted all permissions on them (if the IAuthorizer implementation supports
 +     this).
 +   - SSTable file name is changed. Now you don't have Keyspace/CF name
 +     in file name. Also, secondary index has its own directory under parent's
 +     directory.
 +   - Support for user-defined functions and user-defined aggregates have
 +     been added to CQL.
 +     ************************************************************************
 +     IMPORTANT NOTE: user-defined functions can be used to execute
 +     arbitrary and possibly evil code in Cassandra 2.2, and are
 +     therefore disabled by default.  To enable UDFs edit
 +     cassandra.yaml and set enable_user_defined_functions to true.
 +     CASSANDRA-9402 will add a security manager for UDFs in Cassandra
 +     3.0.  This will inherently be backwards-incompatible with any 2.2
 +     UDF that perform insecure operations such as opening a socket or
 +     writing to the filesystem.
 +     ************************************************************************
 +   - Row-cache is now fully off-heap.
 +   - jemalloc is now automatically preloaded and used on Linux and OS-X if
 +     installed.
 +   - Please ensure on Unix platforms that there is no
 +     installed which is accessible by Cassandra. Old versions of
 +     libjna packages (< 4.0.0) will cause problems - e.g. Debian Wheezy
 +     contains libjna versin 3.2.x.
 +   - The node now keeps up when streaming is failed during bootstrapping. You can
 +     use new `nodetool bootstrap resume` command to continue streaming after resolving
 +     an issue.
 +   - Protocol version 4 specifies that bind variables do not require having a
 +     value when executing a statement. Bind variables without a value are
 +     called 'unset'. The 'unset' bind variable is serialized as the int
 +     value '-2' without following bytes.
 +     In an EXECUTE or BATCH request an unset bind value does not modify the value and
 +     does not create a tombstone, an unset bind ttl is treated as 'unlimited',
 +     an unset bind timestamp is treated as 'now', an unset bind counter operation
 +     does not change the counter value.
 +     Unset tuple field, UDT field and map key are not allowed.
 +     In a QUERY request an unset limit is treated as 'unlimited'.
 +     Unset WHERE clauses with unset partition column, clustering column
 +     or index column are not allowed.
 +   - New `ByteType` (cql tinyint). 1-byte signed integer
 +   - New `ShortType` (cql smallint). 2-byte signed integer
 +   - New `SimpleDateType` (cql date). 4-byte unsigned integer
 +   - New `TimeType` (cql time). 8-byte long
 +   - The toDate(timeuuid), toTimestamp(timeuuid) and toUnixTimestamp(timeuuid) functions
have been added to allow
 +     to convert from timeuuid into date type, timestamp type and bigint raw value.
 +     The functions unixTimestampOf(timeuuid) and dateOf(timeuuid) have been deprecated.
 +   - The toDate(timestamp) and toUnixTimestamp(timestamp) functions have been added to allow
 +     to convert from timestamp into date type and bigint raw value.
 +   - The toTimestamp(date) and toUnixTimestamp(date) functions have been added to allow
 +     to convert from date into timestamp type and bigint raw value.
+ 2.1.9
+ =====
+ Upgrading
+ ---------
+     - cqlsh will now display timestamps with a UTC timezone. Previously,
+       timestamps were displayed with the local timezone.
+     - Commit log files are no longer recycled by default, due to negative
+       performance implications. This can be enabled again with the 
+       commitlog_segment_recycling option in your cassandra.yaml 
+ 2.1.8
+ =====
+ Upgrading
+ ---------
+     - Nothing specific to this release, but please see 2.1 if you are upgrading
+       from a previous version.
+ 2.1.7
+ =====
 -    - Nothing specific to this release, but please see 2.1 if you are upgrading
 -      from a previous version.
 +   - Thrift rpc is no longer being started by default.
 +     Set `start_rpc` parameter to `true` to enable it.
 +   - Pig's CqlStorage has been removed, use CqlNativeStorage instead
 +   - Pig's CassandraStorage has been deprecated. CassandraStorage
 +     should only be used against tables created via thrift.
 +     Use CqlNativeStorage for all other tables.
 +   - IAuthenticator been updated to remove responsibility for user/role
 +     maintenance and is now solely responsible for validating credentials,
 +     This is primarily done via SASL, though an optional method exists for
 +     systems which need support for the Thrift login() method.
 +   - IRoleManager interface has been added which takes over the maintenance
 +     functions from IAuthenticator. IAuthorizer is mainly unchanged. Auth data
 +     in systems using the stock internal implementations PasswordAuthenticator
 +     & CassandraAuthorizer will be automatically converted during upgrade,
 +     with minimal operator intervention required. Custom implementations will
 +     require modification, though these can be used in conjunction with the
 +     stock CassandraRoleManager so providing an IRoleManager implementation
 +     should not usually be necessary.
 +   - Fat client support has been removed since we have push notifications to clients
 +   - cassandra-cli has been removed. Please use cqlsh instead.
 +   - YamlFileNetworkTopologySnitch has been removed; switch to
 +     GossipingPropertyFileSnitch instead.
 +   - CQL2 has been removed entirely in this release (previously deprecated
 +     in 2.0.0). Please switch to CQL3 if you haven't already done so.
 +   - Very large batches will now be rejected (defaults to 50kb). This
 +     can be customized by modifying batch_size_fail_threshold_in_kb.
 +   - The results of CQL3 queries containing an IN restriction will be ordered
 +     in the normal order and not anymore in the order in which the column values were
 +     specified in the IN restriction.
 +   - Some secondary index queries with restrictions on non-indexed clustering
 +     columns were not requiring ALLOW FILTERING as they should. This has been
 +     fixed, and those queries now require ALLOW FILTERING (see CASSANDRA-8418
 +     for details).
 +   - The SSTableSimpleWriter and SSTableSimpleUnsortedWriter classes have been
 +     deprecated and will be removed in the next major Cassandra release. You
 +     should use the CQLSSTableWriter class instead.
 +   - The sstable2json and json2sstable tools have been deprecated and will be
 +     removed in the next major Cassandra release. See CASSANDRA-9618
 +     ( for details.
 +   - nodetool enablehandoff will no longer support a list of data centers starting
 +     with the next major release. Two new commands will be added, enablehintsfordc and disablehintsfordc,
 +     to exclude data centers from using hinted handoff when the global status is enabled.
 +     In cassandra.yaml, hinted_handoff_enabled will no longer support a list of data centers
 +     with the next major release. A new setting will be added, hinted_handoff_disabled_datacenters,
 +     to exclude data centers when the global status is enabled, see CASSANDRA-9035 for details.
diff --cc pylib/cqlshlib/
index 00d5b40,37bd361..f028424
--- a/pylib/cqlshlib/
+++ b/pylib/cqlshlib/
@@@ -14,22 -14,18 +14,24 @@@
  # See the License for the specific language governing permissions and
  # limitations under the License.
 -import sys
 -import re
 -import time
  import calendar
  import math
 +import platform
 +import re
 +import sys
 +import platform
 +import time
  from collections import defaultdict
  from . import wcwidth
  from .displaying import colorme, FormattedValue, DEFAULT_VALUE_COLORS
 +from datetime import datetime, timedelta
  from cassandra.cqltypes import EMPTY
+ from cassandra.util import datetime_from_timestamp
+ from util import UTC
 +is_win = platform.system() == 'Windows'
  unicode_controlchars_re = re.compile(r'[\x00-\x31\x7f-\xa0]')
  controlchars_re = re.compile(r'[\x00-\x31\x7f-\xff]')
@@@ -190,40 -174,12 +192,18 @@@ def format_value_timestamp(val, colorma
          bval = "'%s'" % bval
      return colorme(bval, colormap, 'timestamp')
  def strftime(time_format, seconds):
-     local = time.localtime(seconds)
-     formatted = time.strftime(time_format, local)
-     if local.tm_isdst != 0:
-         offset = -time.altzone
-     else:
-         offset = -time.timezone
-     if not is_win and (formatted[-4:] != '0000' or time_format[-2:] != '%z' or offset ==
-         return formatted
-     elif is_win and time_format[-2:] != '%z':
-         return formatted
-     # deal with %z on platforms where it isn't supported. see CASSANDRA-4746.
-     if offset < 0:
-         sign = '-'
-     else:
-         sign = '+'
-     hours, minutes = divmod(abs(offset) / 60, 60)
-     # Need to strip out invalid %z output on Windows. C libs give us 'Eastern Standard Time'
instead of +/- GMT
-     if is_win and time_format[-2:] == '%z':
-         # Remove chars and strip trailing spaces left behind
-         formatted = re.sub('[A-Za-z]', '', formatted).rstrip()
-         return formatted + sign + '{0:0=2}{1:0=2}'.format(hours, minutes)
-     else:
-         return formatted[:-5] + sign + '{0:0=2}{1:0=2}'.format(hours, minutes)
+     tzless_dt = datetime_from_timestamp(seconds)
+     return tzless_dt.replace(tzinfo=UTC()).strftime(time_format)
 +def format_value_date(val, colormap, **_):
 +    return format_python_formatted_type(val, colormap, 'date')
 +def format_value_time(val, colormap, **_):
 +    return format_python_formatted_type(val, colormap, 'time')
  def format_value_text(val, encoding, colormap, quote=False, **_):
      escapedval = val.replace(u'\\', u'\\\\')

View raw message