Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification.
The "CassandraCli" page has been changed by EricEvans.
The comment on this change is: cleanup.
http://wiki.apache.org/cassandra/CassandraCli?action=diff&rev1=8&rev2=9
--------------------------------------------------
- The CLI is a simple java program that lets you connect to a Cassandra server and interactively
store and retrieve data.
- After starting the Cassandra server, launch the CLI using:
+ Cassandra ships with a very basic interactive command line interface, or shell. Using the
CLI you can connect to remote nodes in the cluster, set and retrieve records and columns,
or query node and cluster meta-data (i.e. cluster name, keyspace listings and disposition,
etc). The CLI is handy for quick tests or for familiarizing yourself with the data-model.
+
+ You can start the CLI using the `bin/cassandra-cli` startup script.
{{{
- ./bin/cassandra-cli -host localhost -port 9160
+ evans@achilles:~/cassandra$ bin/cassandra-cli -host localhost -port 9160
- }}}
-
- You should see:
- {{{
Connected to localhost/9160
Welcome to cassandra CLI.
@@ -23, +20 @@
{{{
cassandra> set Keyspace1.Standard1['jsmith']['first'] = 'John'
Value inserted.
- cassandra> set Keyspace1.Standard1['jsmith']['last'] = 'Simmons'
+ cassandra> set Keyspace1.Standard1['jsmith']['last'] = 'Smith'
Value inserted.
cassandra> set Keyspace1.Standard1['jsmith']['age'] = '42'
Value inserted.
}}}
- We just added a key `jsmith` and three columns (`first`, `last`, and `age`) to the `Standard1`
column family. (Check out DataModel for a primer on the Table abstraction provided by Cassandra
if these terms don't make sense just yet.)
+ In the example above we created a record in the `Keyspace1` keyspace and `Standard1` column
family using the key `jsmith`. This record has three columns, `first`, `last`, and `age`.
Each of these commands is the equivalent to an `insert()` using the [[API|Thrift API]].
- Now we read back the `jsmith` row to see what it contains:
+ Now let's read back the `jsmith` row to see what it contains:
{{{
cassandra> get Keyspace1.Standard1['jsmith']
@@ -42, +39 @@
cassandra>
}}}
+ Note: Using the `get` command in this form is the equivalent to a `get_slice()` using the
[[API|Thrift API]].
+
[ https://issues.apache.org/jira/browse/CASSANDRA-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788353#action_12788353
]
Chris Goffinet commented on CASSANDRA-619:
------------------------------------------
Eric,
That's fine.
> sstable2json makes assumption for keyspace
> ------------------------------------------
>
> Key: CASSANDRA-619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-619
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Reporter: Chris Goffinet
> Assignee: Eric Evans
> Priority: Trivial
> Fix For: 0.5
>
>
> Right now sstable2json makes the assumption that Keyspace folder is one parent above.
When copying SSTables over to another machine to verify some things, it wasn't clear this
is how it worked until looking at source code. It kept complaining about NullPointerException
when trying to lookup an invalid keyspace. Can we be more explicit in this error message and
ability to specify a keyspace name?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788334#action_12788334
]
Eric Evans commented on CASSANDRA-619:
--------------------------------------
The problem is that SSTable/SSTableReader assumes that it can suss out the keyspace name from
the full path to the sstable file (see parseTableName(String)).
Documenting the import/export utilities on the Operations wiki page is on my short list. Do
you think that will suffice?
> sstable2json makes assumption for keyspace
> ------------------------------------------
>
> Key: CASSANDRA-619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-619
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Reporter: Chris Goffinet
> Assignee: Eric Evans
> Priority: Trivial
> Fix For: 0.5
>
>
> Right now sstable2json makes the assumption that Keyspace folder is one parent above.
When copying SSTables over to another machine to verify some things, it wasn't clear this
is how it worked until looking at source code. It kept complaining about NullPointerException
when trying to lookup an invalid keyspace. Can we be more explicit in this error message and
ability to specify a keyspace name?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Add per-keyspace replication factor (possibly even replication strategy)
------------------------------------------------------------------------
Key: CASSANDRA-620
URL: https://issues.apache.org/jira/browse/CASSANDRA-620
Project: Cassandra
Issue Type: New Feature
Reporter: Jonathan Ellis
Fix For: 0.9
(but partitioner may only be cluster-wide, still)
not 100% sure this makes sense but it would allow maintaining system metadata in a replicated-across-entire-cluster
keyspace (without ugly special casing), as well as making Cassandra more flexible as a shared
resource for multiple apps
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788317#action_12788317
]
Jonathan Ellis commented on CASSANDRA-618:
------------------------------------------
Ramzi, could you add a test to SSTableExportTest illustrating the problem?
> json2sstable/sstable2json don't export/import correct column names when the column family
is of BytesType ordering
> ------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-618
> URL: https://issues.apache.org/jira/browse/CASSANDRA-618
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Reporter: Ramzi Rabah
> Priority: Minor
> Fix For: 0.5
>
>
> Easy to reproduce.
> 1- start with an empty node.
> 2- run: client.insert("Keyspace1",
> key_user_id,
> new ColumnPath("Standard1", null, "name".getBytes("UTF-8")),
> "Ramzi".getBytes("UTF-8"),
> timestamp,
> ConsistencyLevel.ONE)
> 3- flush to get sstable
> 4- sstable2json and export the sstable to a file
> 5- delete sstable
> 6- json2sstable and import the json into a new sstable.
> 7- sstable2json on new sstable, you will see that the name is different than the name
in the original json file.
> Also do a get on the column and it will return no result.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-618:
-------------------------------------
Component/s: Tools
Priority: Minor (was: Major)
Affects Version/s: (was: 0.5)
> json2sstable/sstable2json don't export/import correct column names when the column family
is of BytesType ordering
> ------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-618
> URL: https://issues.apache.org/jira/browse/CASSANDRA-618
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Reporter: Ramzi Rabah
> Priority: Minor
> Fix For: 0.5
>
>
> Easy to reproduce.
> 1- start with an empty node.
> 2- run: client.insert("Keyspace1",
> key_user_id,
> new ColumnPath("Standard1", null, "name".getBytes("UTF-8")),
> "Ramzi".getBytes("UTF-8"),
> timestamp,
> ConsistencyLevel.ONE)
> 3- flush to get sstable
> 4- sstable2json and export the sstable to a file
> 5- delete sstable
> 6- json2sstable and import the json into a new sstable.
> 7- sstable2json on new sstable, you will see that the name is different than the name
in the original json file.
> Also do a get on the column and it will return no result.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-619:
-------------------------------------
Affects Version/s: (was: 0.9)
Fix Version/s: (was: 0.9)
0.5
I don't understand the problem -- where else would you put sstables, than under the keyspace
directory?
> sstable2json makes assumption for keyspace
> ------------------------------------------
>
> Key: CASSANDRA-619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-619
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Reporter: Chris Goffinet
> Assignee: Eric Evans
> Priority: Trivial
> Fix For: 0.5
>
>
> Right now sstable2json makes the assumption that Keyspace folder is one parent above.
When copying SSTables over to another machine to verify some things, it wasn't clear this
is how it worked until looking at source code. It kept complaining about NullPointerException
when trying to lookup an invalid keyspace. Can we be more explicit in this error message and
ability to specify a keyspace name?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-619:
-------------------------------------
Component/s: Tools
> sstable2json makes assumption for keyspace
> ------------------------------------------
>
> Key: CASSANDRA-619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-619
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Reporter: Chris Goffinet
> Assignee: Eric Evans
> Priority: Trivial
> Fix For: 0.5
>
>
> Right now sstable2json makes the assumption that Keyspace folder is one parent above.
When copying SSTables over to another machine to verify some things, it wasn't clear this
is how it worked until looking at source code. It kept complaining about NullPointerException
when trying to lookup an invalid keyspace. Can we be more explicit in this error message and
ability to specify a keyspace name?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
sstable2json makes assumption for keyspace
------------------------------------------
Key: CASSANDRA-619
URL: https://issues.apache.org/jira/browse/CASSANDRA-619
Project: Cassandra
Issue Type: Improvement
Affects Versions: 0.9
Reporter: Chris Goffinet
Assignee: Eric Evans
Priority: Trivial
Fix For: 0.9
Right now sstable2json makes the assumption that Keyspace folder is one parent above. When
copying SSTables over to another machine to verify some things, it wasn't clear this is how
it worked until looking at source code. It kept complaining about NullPointerException when
trying to lookup an invalid keyspace. Can we be more explicit in this error message and ability
to specify a keyspace name?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788314#action_12788314
]
Jonathan Ellis commented on CASSANDRA-611:
------------------------------------------
in at least one case there we are only actually using the value, so iterating values makes
even more sense than entryset
> Fix Findbugs: Replace keySet w entrySet
> ---------------------------------------
>
> Key: CASSANDRA-611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-611
> Project: Cassandra
> Issue Type: Bug
> Reporter: Stu Hood
> Assignee: Chris Goffinet
> Attachments: 0001-Replace-keySet-with-entrySet.patch, findbugs-keyset-w-entryset.txt
>
>
> A lot of places in the codebase, we iterate over the keySet() of a Map, and call get()
for each key. For a HashMap, this is still O(N), but when you are dealing with a SortedMap,
this is O(NlogN). The list of occurences are attached.
> Fixing this issue should give us a general (minor?) performance boost.
> In order to find unused method parameters for CASSANDRA-608, I ran FindBugs against Cassandra,
and found a few interesting issues we ought to explore (but not the unused method params,
oi.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Goffinet updated CASSANDRA-611:
-------------------------------------
Attachment: 0001-Replace-keySet-with-entrySet.patch
First stab at replacing some keySet's with entrySet. Working on the getSortedColumn issue
next.
> Fix Findbugs: Replace keySet w entrySet
> ---------------------------------------
>
> Key: CASSANDRA-611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-611
> Project: Cassandra
> Issue Type: Bug
> Reporter: Stu Hood
> Assignee: Chris Goffinet
> Attachments: 0001-Replace-keySet-with-entrySet.patch, findbugs-keyset-w-entryset.txt
>
>
> A lot of places in the codebase, we iterate over the keySet() of a Map, and call get()
for each key. For a HashMap, this is still O(N), but when you are dealing with a SortedMap,
this is O(NlogN). The list of occurences are attached.
> Fixing this issue should give us a general (minor?) performance boost.
> In order to find unused method parameters for CASSANDRA-608, I ran FindBugs against Cassandra,
and found a few interesting issues we ought to explore (but not the unused method params,
oi.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
json2sstable/sstable2json don't export/import correct column names when the column family is
of BytesType ordering
------------------------------------------------------------------------------------------------------------------
Key: CASSANDRA-618
URL: https://issues.apache.org/jira/browse/CASSANDRA-618
Project: Cassandra
Issue Type: Bug
Affects Versions: 0.5
Reporter: Ramzi Rabah
Fix For: 0.5
Easy to reproduce.
1- start with an empty node.
2- run: client.insert("Keyspace1",
key_user_id,
new ColumnPath("Standard1", null, "name".getBytes("UTF-8")),
"Ramzi".getBytes("UTF-8"),
timestamp,
ConsistencyLevel.ONE)
3- flush to get sstable
4- sstable2json and export the sstable to a file
5- delete sstable
6- json2sstable and import the json into a new sstable.
7- sstable2json on new sstable, you will see that the name is different than the name in the
original json file.
Also do a get on the column and it will return no result.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-564.
--------------------------------------
Resolution: Fixed
Assignee: Jaakko Laine
committed
> Provide recoverability when a node dies and it is impossible to get the same IP.
> --------------------------------------------------------------------------------
>
> Key: CASSANDRA-564
> URL: https://issues.apache.org/jira/browse/CASSANDRA-564
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Anthony Molinaro
> Assignee: Jaakko Laine
> Priority: Minor
> Fix For: 0.5
>
> Attachments: 564-1-replace-bootstrapmetadata-with-streamrequest.patch, 564-2-add-removetoken-command.patch
>
>
> From the descriptions on the mailing list, when a node dies permanently from hardware
or other failure and you need to replace it, it must have the same IP. For people running
in cloud environments, this is often times impossible. So it would be very useful if there
was a way to replace a node with a new node without requiring the same IP.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification.
The "API" page has been changed by JonathanEllis.
http://wiki.apache.org/cassandra/API?action=diff&rev1=28&rev2=29
--------------------------------------------------
Counts the columns present in `column_parent`.
+ get_range_slice
+
+ `list<KeySlice> get_range_slice(keyspace, column_parent, predicate, start_key,
finish_key, row_count=100, consistency_level)`
+
+ Replaces get_key_range. Returns a list of slices, sorted by row key, starting with start,
ending with finish (both inclusive) and at most count long. The empty string ("") can be used
as a sentinel value to get the first/last existing key (or first/last column in the column
predicate parameter). Unlike get_key_range, this applies the given predicate to all keys in
the range, not just those with undeleted matching data. This method is only allowed when
using an order-preserving partitioner.
+
=== get_key_range ===
`list<string> get_key_range(keyspace, column_family, start, finish, count=100, consistency_level)`
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "Operations" page has been changed by JonathanEllis. The comment on this change is: nodeprobe removetoken. http://wiki.apache.org/cassandra/Operations?action=diff&rev1=10&rev2=11 -------------------------------------------------- Cassandra is smart enough to transfer data from the nearest source node(s), if your !EndpointSnitch is configured correctly. So, the new node doesn't need to be in the same datacenter as the primary replica for the Range it is bootstrapping into, as long as another replica is in the datacenter with the new one. == Removing nodes entirely == - You can take a node out of the cluster with `nodeprobe decommission.` The node must be live at decommission time (until CASSANDRA-564 is done). + You can take a node out of the cluster with `nodeprobe decommission` to a live node, or `nodeprobe removetoken` to remove a dead one. Again, no data is removed automatically, so if you want to put the node back into service and you don't need the data on it anymore, it should be removed manually.
[ https://issues.apache.org/jira/browse/CASSANDRA-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Goffinet reassigned CASSANDRA-611:
----------------------------------------
Assignee: Chris Goffinet
> Fix Findbugs: Replace keySet w entrySet
> ---------------------------------------
>
> Key: CASSANDRA-611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-611
> Project: Cassandra
> Issue Type: Bug
> Reporter: Stu Hood
> Assignee: Chris Goffinet
> Attachments: findbugs-keyset-w-entryset.txt
>
>
> A lot of places in the codebase, we iterate over the keySet() of a Map, and call get()
for each key. For a HashMap, this is still O(N), but when you are dealing with a SortedMap,
this is O(NlogN). The list of occurences are attached.
> Fixing this issue should give us a general (minor?) performance boost.
> In order to find unused method parameters for CASSANDRA-608, I ran FindBugs against Cassandra,
and found a few interesting issues we ought to explore (but not the unused method params,
oi.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Author: eevans
Date: Wed Dec 9 18:46:07 2009
New Revision: 888914
URL: http://svn.apache.org/viewvc?rev=888914&view=rev
Log:
tagging cassandra-0.5.0-beta2
Added:
incubator/cassandra/tags/cassandra-0.5.0-beta2/
- copied from r888873, incubator/cassandra/branches/cassandra-0.5/
incubator/cassandra/tags/cassandra-0.5.0-beta2/build.xml
- copied unchanged from r888913, incubator/cassandra/branches/cassandra-0.5/build.xml
incubator/cassandra/tags/cassandra-0.5.0-beta2/interface/cassandra.thrift
- copied unchanged from r888913, incubator/cassandra/branches/cassandra-0.5/interface/cassandra.thrift
incubator/cassandra/tags/cassandra-0.5.0-beta2/interface/gen-java/org/apache/cassandra/service/Constants.java
- copied unchanged from r888913, incubator/cassandra/branches/cassandra-0.5/interface/gen-java/org/apache/cassandra/service/Constants.java
incubator/cassandra/tags/cassandra-0.5.0-beta2/test/unit/org/apache/cassandra/service/ReadResponseResolverTest.java
- copied unchanged from r888912, incubator/cassandra/branches/cassandra-0.5/test/unit/org/apache/cassandra/service/ReadResponseResolverTest.java
Author: eevans
Date: Wed Dec 9 18:45:00 2009
New Revision: 888913
URL: http://svn.apache.org/viewvc?rev=888913&view=rev
Log:
updated versioning for 0.5.0-beta2 release
Modified:
incubator/cassandra/branches/cassandra-0.5/build.xml
incubator/cassandra/branches/cassandra-0.5/interface/cassandra.thrift
incubator/cassandra/branches/cassandra-0.5/interface/gen-java/org/apache/cassandra/service/Constants.java
Modified: incubator/cassandra/branches/cassandra-0.5/build.xml
URL: http://svn.apache.org/viewvc/incubator/cassandra/branches/cassandra-0.5/build.xml?rev=888913&r1=888912&r2=888913&view=diff
==============================================================================
--- incubator/cassandra/branches/cassandra-0.5/build.xml (original)
+++ incubator/cassandra/branches/cassandra-0.5/build.xml Wed Dec 9 18:45:00 2009
@@ -37,7 +37,7 @@
<property name="test.name" value="*Test"/>
<property name="test.unit.src" value="${test.dir}/unit"/>
<property name="dist.dir" value="${build.dir}/dist"/>
- <property name="version" value="0.4.2"/>
+ <property name="version" value="0.5.0-beta2"/>
<property name="final.name" value="${ant.project.name}-${version}"/>
<!-- http://cobertura.sourceforge.net/ -->
Modified: incubator/cassandra/branches/cassandra-0.5/interface/cassandra.thrift
URL: http://svn.apache.org/viewvc/incubator/cassandra/branches/cassandra-0.5/interface/cassandra.thrift?rev=888913&r1=888912&r2=888913&view=diff
==============================================================================
--- incubator/cassandra/branches/cassandra-0.5/interface/cassandra.thrift (original)
+++ incubator/cassandra/branches/cassandra-0.5/interface/cassandra.thrift Wed Dec 9 18:45:00
2009
@@ -40,7 +40,7 @@
# no promises are made other than "if both are equal, you're good."
# in particular, don't try to parse numeric information out and assume
# that a "greater" version is a superset of a "smaller" one.
-const string VERSION = "0.5-beta1"
+const string VERSION = "0.5-beta2"
#
Modified: incubator/cassandra/branches/cassandra-0.5/interface/gen-java/org/apache/cassandra/service/Constants.java
URL: http://svn.apache.org/viewvc/incubator/cassandra/branches/cassandra-0.5/interface/gen-java/org/apache/cassandra/service/Constants.java?rev=888913&r1=888912&r2=888913&view=diff
==============================================================================
--- incubator/cassandra/branches/cassandra-0.5/interface/gen-java/org/apache/cassandra/service/Constants.java
(original)
+++ incubator/cassandra/branches/cassandra-0.5/interface/gen-java/org/apache/cassandra/service/Constants.java
Wed Dec 9 18:45:00 2009
@@ -40,6 +40,6 @@
public class Constants {
- public static final String VERSION = "0.5-beta1";
+ public static final String VERSION = "0.5-beta2";
}
Author: eevans
Date: Wed Dec 9 18:43:20 2009
New Revision: 888912
URL: http://svn.apache.org/viewvc?rev=888912&view=rev
Log:
prepend apache license text (missing from new file)
Modified:
incubator/cassandra/branches/cassandra-0.5/test/unit/org/apache/cassandra/service/ReadResponseResolverTest.java
Modified: incubator/cassandra/branches/cassandra-0.5/test/unit/org/apache/cassandra/service/ReadResponseResolverTest.java
URL: http://svn.apache.org/viewvc/incubator/cassandra/branches/cassandra-0.5/test/unit/org/apache/cassandra/service/ReadResponseResolverTest.java?rev=888912&r1=888911&r2=888912&view=diff
==============================================================================
--- incubator/cassandra/branches/cassandra-0.5/test/unit/org/apache/cassandra/service/ReadResponseResolverTest.java
(original)
+++ incubator/cassandra/branches/cassandra-0.5/test/unit/org/apache/cassandra/service/ReadResponseResolverTest.java
Wed Dec 9 18:43:20 2009
@@ -1,4 +1,25 @@
package org.apache.cassandra.service;
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
import java.util.Arrays;
[ https://issues.apache.org/jira/browse/CASSANDRA-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788212#action_12788212
]
Stu Hood commented on CASSANDRA-611:
------------------------------------
I just noticed that this report missed the most critical of all our abuses of keySet(): calling
ColumnFamily.getSortedValues(), and then calling getColumn(name) on another CF. For two sorted
lists, we should iterate in parallel.
> Fix Findbugs: Replace keySet w entrySet
> ---------------------------------------
>
> Key: CASSANDRA-611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-611
> Project: Cassandra
> Issue Type: Bug
> Reporter: Stu Hood
> Attachments: findbugs-keyset-w-entryset.txt
>
>
> A lot of places in the codebase, we iterate over the keySet() of a Map, and call get()
for each key. For a HashMap, this is still O(N), but when you are dealing with a SortedMap,
this is O(NlogN). The list of occurences are attached.
> Fixing this issue should give us a general (minor?) performance boost.
> In order to find unused method parameters for CASSANDRA-608, I ran FindBugs against Cassandra,
and found a few interesting issues we ought to explore (but not the unused method params,
oi.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Author: eevans
Date: Wed Dec 9 16:57:54 2009
New Revision: 888873
URL: http://svn.apache.org/viewvc?rev=888873&view=rev
Log:
branch for cassandra-0.5
Added:
incubator/cassandra/branches/cassandra-0.5/
- copied from r888871, incubator/cassandra/trunk/
Author: jbellis
Date: Wed Dec 9 16:42:07 2009
New Revision: 888870
URL: http://svn.apache.org/viewvc?rev=888870&view=rev
Log:
update comments
Modified:
incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java
Modified: incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java
URL: http://svn.apache.org/viewvc/incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java?rev=888870&r1=888869&r2=888870&view=diff
==============================================================================
--- incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java (original)
+++ incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java Wed
Dec 9 16:42:07 2009
@@ -286,7 +286,7 @@
}
/**
- * Read the data from one replica. If there is no reply, read the data from another.
In the event we get
+ * Read the data from one replica. When we get
* the data we perform consistency checks and figure out if any repairs need to be done
to the replicas.
* @param commands a set of commands to perform reads
* @return the row associated with command.key
@@ -484,10 +484,7 @@
}
/*
- * This function executes the read protocol locally and should be used only if consistency
is not a concern.
- * Read the data from the local disk and return if the row is NOT NULL. If the data is
NULL do the read from
- * one of the other replicas (in the same data center if possible) till we get the data.
In the event we get
- * the data we perform consistency checks and figure out if any repairs need to be done
to the replicas.
+ * This function executes the read protocol locally. Consistency checks are performed
in the background.
*/
private static List<Row> weakReadLocal(List<ReadCommand> commands)
{
[ https://issues.apache.org/jira/browse/CASSANDRA-16?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-16:
------------------------------------
Fix Version/s: 0.9
Re-scheduling for 0.9. Maybe this time for sure. :)
> Memory efficient compactions
> -----------------------------
>
> Key: CASSANDRA-16
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: All
> Reporter: Sandeep Tata
> Fix For: 0.9
>
>
> The basic idea is to allow rows to get large enough that they don't have to fit in memory
entirely, but can easily fit on a disk. The compaction algorithm today de-serializes the entire
row in memory before writing out the compacted SSTable (see ColumnFamilyStore.doCompaction()
and associated methods).
> The requirement is to have a compaction method with a lower memory requirement so we
can support rows larger than available main memory. To re-use the old FB example, if we stored
a user's inbox in a row, we'd want the inbox to grow bigger than memory so long as it fit
on disk.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Author: jbellis
Date: Wed Dec 9 16:37:03 2009
New Revision: 888866
URL: http://svn.apache.org/viewvc?rev=888866&view=rev
Log:
Add removeToken command to StorageService and nodeprobe. patch by Jaakko Laine; reviewed
by jbellis for CASSANDRA-564
Modified:
incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageServiceMBean.java
incubator/cassandra/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java
Modified: incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
URL: http://svn.apache.org/viewvc/incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java?rev=888866&r1=888865&r2=888866&view=diff
==============================================================================
--- incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java (original)
+++ incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java Wed
Dec 9 16:37:03 2009
@@ -68,6 +68,10 @@
public final static String STATE_LEAVING = "LEAVING";
public final static String STATE_LEFT = "LEFT";
+ private final static char StateDelimiter = ',';
+ private final static String REMOVE_TOKEN = "remove";
+ private final static String LEFT_NORMALLY = "left";
+
/* All verb handler identifiers */
public final static String mutationVerbHandler_ = "ROW-MUTATION-VERB-HANDLER";
public final static String binaryVerbHandler_ = "BINARY-VERB-HANDLER";
@@ -433,13 +437,107 @@
}
else if (STATE_LEFT.equals(stateName))
{
- Token token = getPartitioner().getTokenFactory().fromString(state.getValue());
- assert tokenMetadata_.getToken(endpoint).equals(token);
- tokenMetadata_.removeEndpoint(endpoint);
+ // STATE_LEFT state is of form (REMOVE_TOKEN|LEFT_NORMALLY)<StateDelimiter><token>
+ String stateValue = state.getValue();
+ int index = stateValue.indexOf(StateDelimiter);
+ assert (index != -1);
+ String typeOfState = stateValue.substring(0, index);
+ Token token = getPartitioner().getTokenFactory().fromString(stateValue.substring(index
+ 1));
+
+ if (typeOfState.equals(LEFT_NORMALLY))
+ {
+ if (tokenMetadata_.isMember(endpoint))
+ {
+ if (logger_.isDebugEnabled())
+ logger_.debug(endpoint + " state left, token " + token);
+ assert tokenMetadata_.getToken(endpoint).equals(token);
+ tokenMetadata_.removeEndpoint(endpoint);
+ }
+ }
+ else
+ {
+ assert (typeOfState.equals(REMOVE_TOKEN));
+ InetAddress endPointThatLeft = tokenMetadata_.getEndPoint(token);
+ if (logger_.isDebugEnabled())
+ logger_.debug("Token " + token + " removed manually (endpoint was " +
((endPointThatLeft == null) ? "unknown" : endPointThatLeft) + ")");
+ if (endPointThatLeft != null)
+ {
+ restoreReplicaCount(endPointThatLeft);
+ tokenMetadata_.removeEndpoint(endPointThatLeft);
+ }
+ }
+
replicationStrategy_.removeObsoletePendingRanges();
}
}
+ /**
+ * Called when endPoint is removed from the ring without proper
+ * STATE_LEAVING -> STATE_LEFT sequence. This function checks
+ * whether this node becomes responsible for new ranges as a
+ * consequence and streams data if needed.
+ *
+ * This is rather ineffective, but it does not matter so much
+ * since this is called very seldom
+ *
+ * @param endPoint node that has left
+ */
+ private void restoreReplicaCount(InetAddress endPoint)
+ {
+ InetAddress myAddress = FBUtilities.getLocalAddress();
+
+ // get all ranges that change ownership (that is, a node needs
+ // to take responsibility for new range)
+ Multimap<Range, InetAddress> changedRanges = getChangedRangesForLeaving(endPoint);
+
+ // check if any of these ranges are coming our way
+ Set<Range> myNewRanges = new HashSet<Range>();
+ for (Map.Entry<Range, InetAddress> entry : changedRanges.entries())
+ {
+ if (entry.getValue().equals(myAddress))
+ myNewRanges.add(entry.getKey());
+ }
+
+ if (!myNewRanges.isEmpty())
+ {
+ if (logger_.isDebugEnabled())
+ logger_.debug(endPoint + " was removed, my added ranges: " + StringUtils.join(myNewRanges,
", "));
+
+ Multimap<Range, InetAddress> rangeAddresses = replicationStrategy_.getRangeAddresses(tokenMetadata_);
+ Multimap<InetAddress, Range> sourceRanges = HashMultimap.create();
+ IFailureDetector failureDetector = FailureDetector.instance();
+
+ // find alive sources for our new ranges
+ for (Range myNewRange : myNewRanges)
+ {
+ List<InetAddress> sources = DatabaseDescriptor.getEndPointSnitch().sortByProximity(myAddress,
rangeAddresses.get(myNewRange));
+
+ assert (!sources.contains(myAddress));
+
+ for (InetAddress source : sources)
+ {
+ if (source.equals(endPoint))
+ continue;
+
+ if (failureDetector.isAlive(source))
+ {
+ sourceRanges.put(source, myNewRange);
+ break;
+ }
+ }
+ }
+
+ // Finally we have a list of addresses and ranges to
+ // stream. Proceed to stream
+ for (Map.Entry<InetAddress, Collection<Range>> entry : sourceRanges.asMap().entrySet())
+ {
+ if (logger_.isDebugEnabled())
+ logger_.debug("Requesting from " + entry.getKey() + " ranges " + StringUtils.join(entry.getValue(),
", "));
+ Streaming.requestRanges(entry.getKey(), entry.getValue());
+ }
+ }
+ }
+
private Multimap<Range, InetAddress> getChangedRangesForLeaving(InetAddress endpoint)
{
// First get all ranges the leaving endpoint is responsible for
@@ -1033,7 +1131,7 @@
if (logger_.isDebugEnabled())
logger_.debug("");
- Gossiper.instance().addApplicationState(STATE_LEFT, new ApplicationState(getLocalToken().toString()));
+ Gossiper.instance().addApplicationState(STATE_LEFT, new ApplicationState(LEFT_NORMALLY
+ StateDelimiter + getLocalToken().toString()));
try
{
Thread.sleep(2 * Gossiper.intervalInMillis_);
@@ -1132,6 +1230,37 @@
unbootstrap(finishMoving);
}
+ public void removeToken(String tokenString)
+ {
+ Token token = partitioner_.getTokenFactory().fromString(tokenString);
+
+ // Here we could refuse the operation from continuing if we
+ // cannot find the endpoint for this token from metadata, but
+ // that would prevent this command from being issued by a node
+ // that has never seen the failed node.
+ InetAddress endPoint = tokenMetadata_.getEndPoint(token);
+ if (endPoint != null)
+ {
+ // Let's make sure however that we're not removing a live
+ // token (member)
+ if (Gossiper.instance().getLiveMembers().contains(endPoint))
+ throw new UnsupportedOperationException("Node " + endPoint + " is alive and
owns this token. Use decommission command to remove it from the ring");
+
+ restoreReplicaCount(endPoint);
+ tokenMetadata_.removeEndpoint(endPoint);
+ replicationStrategy_.removeObsoletePendingRanges();
+ }
+
+ // This is not the cleanest way as we're adding STATE_LEFT for
+ // a foreign token to our own EP state. Another way would be
+ // to add new AP state for this command, but that would again
+ // increase the amount of data to be gossiped in the cluster -
+ // not good. REMOVE_TOKEN|LEFT_NORMALLY is used to distinguish
+ // between removetoken command and normal state left, so it is
+ // not so bad.
+ Gossiper.instance().addApplicationState(STATE_LEFT, new ApplicationState(REMOVE_TOKEN
+ StateDelimiter + token.toString()));
+ }
+
public WriteResponseHandler getWriteResponseHandler(int blockFor, int consistency_level)
{
return replicationStrategy_.getWriteResponseHandler(blockFor, consistency_level);
Modified: incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageServiceMBean.java
URL: http://svn.apache.org/viewvc/incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageServiceMBean.java?rev=888866&r1=888865&r2=888866&view=diff
==============================================================================
--- incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageServiceMBean.java
(original)
+++ incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageServiceMBean.java
Wed Dec 9 16:37:03 2009
@@ -147,6 +147,12 @@
*/
public void cancelPendingRanges();
+ /**
+ * removeToken removes token (and all data associated with
+ * enpoint that had it) from the ring
+ */
+ public void removeToken(String token);
+
/** set the logging level at runtime */
public void setLog4jLevel(String classQualifier, String level);
}
Modified: incubator/cassandra/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java
URL: http://svn.apache.org/viewvc/incubator/cassandra/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java?rev=888866&r1=888865&r2=888866&view=diff
==============================================================================
--- incubator/cassandra/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java (original)
+++ incubator/cassandra/trunk/src/java/org/apache/cassandra/tools/NodeProbe.java Wed Dec
9 16:37:03 2009
@@ -403,6 +403,11 @@
ssProxy.cancelPendingRanges();
}
+ public void removeToken(String token)
+ {
+ ssProxy.removeToken(token);
+ }
+
/**
* Print out the size of the queues in the thread pools
*
@@ -498,7 +503,7 @@
HelpFormatter hf = new HelpFormatter();
String header = String.format(
"%nAvailable commands: ring, info, cleanup, compact, cfstats, snapshot [name],
clearsnapshot, " +
- "tpstats, flush, repair, decommission, move, loadbalance, cancelpending,
" +
+ "tpstats, flush, repair, decommission, move, loadbalance, cancelpending,
removetoken, " +
" getcompactionthreshold, setcompactionthreshold [minthreshold] ([maxthreshold])");
String usage = String.format("java %s -host <arg> <command>%n", NodeProbe.class.getName());
hf.printHelp(usage, "", options, header);
@@ -577,6 +582,14 @@
{
probe.cancelPendingRanges();
}
+ else if (cmdName.equals("removetoken"))
+ {
+ if (arguments.length <= 1)
+ {
+ System.err.println("missing token argument");
+ }
+ probe.removeToken(arguments[1]);
+ }
else if (cmdName.equals("snapshot"))
{
String snapshotName = "";
Author: jbellis
Date: Wed Dec 9 16:36:31 2009
New Revision: 888864
URL: http://svn.apache.org/viewvc?rev=888864&view=rev
Log:
Rename BootstrapMetadata and friends to StreamRequest* as that is what those essentially are.
Move them to cassandra.io where rest of streaming stuff resides. patch by Jaakko Laine; reviewed
by jbellis for CASSANDRA-564
Added:
incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestMessage.java
(with props)
incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestMetadata.java
(contents, props changed)
- copied, changed from r888712, incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootstrapMetadata.java
incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestVerbHandler.java
(contents, props changed)
- copied, changed from r888712, incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootstrapMetadataVerbHandler.java
Removed:
incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootstrapMetadata.java
incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootstrapMetadataMessage.java
incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootstrapMetadataVerbHandler.java
Modified:
incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java
incubator/cassandra/trunk/src/java/org/apache/cassandra/io/Streaming.java
incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java
incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
Modified: incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java
URL: http://svn.apache.org/viewvc/incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java?rev=888864&r1=888863&r2=888864&view=diff
==============================================================================
--- incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java (original)
+++ incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java Wed Dec
9 16:36:31 2009
@@ -33,21 +33,13 @@
import org.apache.cassandra.locator.TokenMetadata;
import org.apache.cassandra.locator.AbstractReplicationStrategy;
import org.apache.cassandra.net.*;
- import org.apache.cassandra.net.io.StreamContextManager;
- import org.apache.cassandra.net.io.IStreamComplete;
import org.apache.cassandra.service.StorageService;
- import org.apache.cassandra.service.StreamManager;
- import org.apache.cassandra.utils.LogUtil;
import org.apache.cassandra.utils.SimpleCondition;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.gms.FailureDetector;
import org.apache.cassandra.gms.IFailureDetector;
- import org.apache.cassandra.io.DataInputBuffer;
- import org.apache.cassandra.io.SSTableReader;
- import org.apache.cassandra.io.SSTableWriter;
- import org.apache.cassandra.db.ColumnFamilyStore;
- import org.apache.cassandra.db.Table;
+ import org.apache.cassandra.io.Streaming;
import com.google.common.collect.Multimap;
import com.google.common.collect.ArrayListMultimap;
@@ -56,7 +48,7 @@
* This class handles the bootstrapping responsibilities for the local endpoint.
*
* - bootstrapTokenVerb asks the most-loaded node what Token to use to split its Range
in two.
- * - bootstrapMetadataVerb tells source nodes to send us the necessary Ranges
+ * - streamRequestVerb tells source nodes to send us the necessary Ranges
* - source nodes send streamInitiateVerb to us to say "get ready to receive data" [if
there is data to send]
* - when we have everything set up to receive the data, we send streamInitiateDoneVerb
back to the source nodes and they start streaming
* - when streaming is complete, we send streamFinishedVerb to the source so it can clean
up on its end
@@ -96,12 +88,10 @@
for (Map.Entry<InetAddress, Collection<Range>> entry : getWorkMap(rangesWithSourceTarget).asMap().entrySet())
{
InetAddress source = entry.getKey();
- if (logger.isDebugEnabled())
- logger.debug("Sending BootstrapMetadataMessage to " + source + "
for " + StringUtils.join(entry.getValue(), ", "));
- BootstrapMetadata bsMetadata = new BootstrapMetadata(address, entry.getValue());
- Message message = BootstrapMetadataMessage.makeBootstrapMetadataMessage(new
BootstrapMetadataMessage(bsMetadata));
- MessagingService.instance().sendOneWay(message, source);
StorageService.instance().addBootstrapSource(source);
+ if (logger.isDebugEnabled())
+ logger.debug("Requesting from " + source + " ranges " + StringUtils.join(entry.getValue(),
", "));
+ Streaming.requestRanges(source, entry.getValue());
}
}
}).start();
Added: incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestMessage.java
URL: http://svn.apache.org/viewvc/incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestMessage.java?rev=888864&view=auto
==============================================================================
--- incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestMessage.java (added)
+++ incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestMessage.java Wed
Dec 9 16:36:31 2009
@@ -0,0 +1,93 @@
+ /**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.io;
+
+import java.io.*;
+
+import org.apache.cassandra.net.Message;
+import org.apache.cassandra.service.StorageService;
+import org.apache.cassandra.utils.FBUtilities;
+import org.apache.cassandra.concurrent.StageManager;
+
+ /**
+ * This class encapsulates the message that needs to be sent to nodes
+ * that handoff data. The message contains information about ranges
+ * that need to be transferred and the target node.
+*/
+class StreamRequestMessage
+{
+ private static ICompactSerializer<StreamRequestMessage> serializer_;
+ static
+ {
+ serializer_ = new StreamRequestMessageSerializer();
+ }
+
+ protected static ICompactSerializer<StreamRequestMessage> serializer()
+ {
+ return serializer_;
+ }
+
+ protected static Message makeStreamRequestMessage(StreamRequestMessage streamRequestMessage)
+ {
+ ByteArrayOutputStream bos = new ByteArrayOutputStream();
+ DataOutputStream dos = new DataOutputStream(bos);
+ try
+ {
+ StreamRequestMessage.serializer().serialize(streamRequestMessage, dos);
+ }
+ catch (IOException e)
+ {
+ throw new IOError(e);
+ }
+ return new Message(FBUtilities.getLocalAddress(), StageManager.streamStage_, StorageService.streamRequestVerbHandler_,
bos.toByteArray() );
+ }
+
+ protected StreamRequestMetadata[] streamRequestMetadata_ = new StreamRequestMetadata[0];
+
+ // TODO only actually ever need one BM, not an array
+ StreamRequestMessage(StreamRequestMetadata... streamRequestMetadata)
+ {
+ assert streamRequestMetadata != null;
+ streamRequestMetadata_ = streamRequestMetadata;
+ }
+}
+
+class StreamRequestMessageSerializer implements ICompactSerializer<StreamRequestMessage>
+{
+ public void serialize(StreamRequestMessage streamRequestMessage, DataOutputStream dos)
throws IOException
+ {
+ StreamRequestMetadata[] streamRequestMetadata = streamRequestMessage.streamRequestMetadata_;
+ dos.writeInt(streamRequestMetadata.length);
+ for (StreamRequestMetadata bsmd : streamRequestMetadata)
+ {
+ StreamRequestMetadata.serializer().serialize(bsmd, dos);
+ }
+ }
+
+ public StreamRequestMessage deserialize(DataInputStream dis) throws IOException
+ {
+ int size = dis.readInt();
+ StreamRequestMetadata[] streamRequestMetadata = new StreamRequestMetadata[size];
+ for ( int i = 0; i < size; ++i )
+ {
+ streamRequestMetadata[i] = StreamRequestMetadata.serializer().deserialize(dis);
+ }
+ return new StreamRequestMessage(streamRequestMetadata);
+ }
+}
Propchange: incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestMessage.java
------------------------------------------------------------------------------
svn:eol-style = native
Copied: incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestMetadata.java
(from r888712, incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootstrapMetadata.java)
URL: http://svn.apache.org/viewvc/incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestMetadata.java?p2=incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestMetadata.java&p1=incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootstrapMetadata.java&r1=888712&r2=888864&rev=888864&view=diff
==============================================================================
--- incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootstrapMetadata.java (original)
+++ incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestMetadata.java
Wed Dec 9 16:36:31 2009
@@ -16,37 +16,34 @@
* limitations under the License.
*/
-package org.apache.cassandra.dht;
+package org.apache.cassandra.io;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
+import java.net.InetAddress;
import java.util.ArrayList;
import java.util.List;
import java.util.Collection;
-import org.apache.cassandra.io.ICompactSerializer;
import org.apache.cassandra.net.CompactEndPointSerializationHelper;
-import java.net.InetAddress;
-
-
+import org.apache.cassandra.dht.Range;
/**
- * This encapsulates information of the list of
- * ranges that a target node requires in order to
- * be bootstrapped. This will be bundled in a
- * BootstrapMetadataMessage and sent to nodes that
- * are going to handoff the data.
+ * This encapsulates information of the list of ranges that a target
+ * node requires to be transferred. This will be bundled in a
+ * StreamRequestsMessage and sent to nodes that are going to handoff
+ * the data.
*/
-class BootstrapMetadata
+class StreamRequestMetadata
{
- private static ICompactSerializer<BootstrapMetadata> serializer_;
+ private static ICompactSerializer<StreamRequestMetadata> serializer_;
static
{
- serializer_ = new BootstrapMetadataSerializer();
+ serializer_ = new StreamRequestMetadataSerializer();
}
- protected static ICompactSerializer<BootstrapMetadata> serializer()
+ protected static ICompactSerializer<StreamRequestMetadata> serializer()
{
return serializer_;
}
@@ -54,7 +51,7 @@
protected InetAddress target_;
protected Collection<Range> ranges_;
- BootstrapMetadata(InetAddress target, Collection<Range> ranges)
+ StreamRequestMetadata(InetAddress target, Collection<Range> ranges)
{
target_ = target;
ranges_ = ranges;
@@ -74,19 +71,19 @@
}
}
-class BootstrapMetadataSerializer implements ICompactSerializer<BootstrapMetadata>
+class StreamRequestMetadataSerializer implements ICompactSerializer<StreamRequestMetadata>
{
- public void serialize(BootstrapMetadata bsMetadata, DataOutputStream dos) throws IOException
+ public void serialize(StreamRequestMetadata srMetadata, DataOutputStream dos) throws
IOException
{
- CompactEndPointSerializationHelper.serialize(bsMetadata.target_, dos);
- dos.writeInt(bsMetadata.ranges_.size());
- for (Range range : bsMetadata.ranges_)
+ CompactEndPointSerializationHelper.serialize(srMetadata.target_, dos);
+ dos.writeInt(srMetadata.ranges_.size());
+ for (Range range : srMetadata.ranges_)
{
Range.serializer().serialize(range, dos);
}
}
- public BootstrapMetadata deserialize(DataInputStream dis) throws IOException
+ public StreamRequestMetadata deserialize(DataInputStream dis) throws IOException
{
InetAddress target = CompactEndPointSerializationHelper.deserialize(dis);
int size = dis.readInt();
@@ -95,7 +92,7 @@
{
ranges.add(Range.serializer().deserialize(dis));
}
- return new BootstrapMetadata( target, ranges );
+ return new StreamRequestMetadata( target, ranges );
}
}
Propchange: incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestMetadata.java
------------------------------------------------------------------------------
svn:eol-style = native
Copied: incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestVerbHandler.java
(from r888712, incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootstrapMetadataVerbHandler.java)
URL: http://svn.apache.org/viewvc/incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestVerbHandler.java?p2=incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestVerbHandler.java&p1=incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootstrapMetadataVerbHandler.java&r1=888712&r2=888864&rev=888864&view=diff
==============================================================================
--- incubator/cassandra/trunk/src/java/org/apache/cassandra/dht/BootstrapMetadataVerbHandler.java
(original)
+++ incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestVerbHandler.java
Wed Dec 9 16:36:31 2009
@@ -16,7 +16,7 @@
* limitations under the License.
*/
-package org.apache.cassandra.dht;
+package org.apache.cassandra.io;
import java.io.File;
import java.io.IOException;
@@ -27,9 +27,6 @@
import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.db.Table;
-import org.apache.cassandra.io.DataInputBuffer;
-import org.apache.cassandra.io.SSTableReader;
-import org.apache.cassandra.io.Streaming;
import java.net.InetAddress;
import org.apache.cassandra.net.IVerbHandler;
@@ -43,34 +40,31 @@
import org.apache.commons.lang.StringUtils;
/**
- * This verb handler handles the BootstrapMetadataMessage that is sent
- * by the leader to the nodes that are responsible for handing off data.
+ * This verb handler handles the StreamRequestMessage that is sent by
+ * the node requesting range transfer.
*/
-public class BootstrapMetadataVerbHandler implements IVerbHandler
+public class StreamRequestVerbHandler implements IVerbHandler
{
- private static Logger logger_ = Logger.getLogger(BootstrapMetadataVerbHandler.class);
+ private static Logger logger_ = Logger.getLogger(StreamRequestVerbHandler.class);
public void doVerb(Message message)
{
if (logger_.isDebugEnabled())
- logger_.debug("Received a BootstrapMetadataMessage from " + message.getFrom());
-
- /* Cannot bootstrap another node if I'm in bootstrap mode myself! */
- assert !StorageService.instance().isBootstrapMode();
+ logger_.debug("Received a StreamRequestMessage from " + message.getFrom());
byte[] body = message.getMessageBody();
DataInputBuffer bufIn = new DataInputBuffer();
bufIn.reset(body, body.length);
try
{
- BootstrapMetadataMessage bsMetadataMessage = BootstrapMetadataMessage.serializer().deserialize(bufIn);
- BootstrapMetadata[] bsMetadata = bsMetadataMessage.bsMetadata_;
+ StreamRequestMessage streamRequestMessage = StreamRequestMessage.serializer().deserialize(bufIn);
+ StreamRequestMetadata[] streamRequestMetadata = streamRequestMessage.streamRequestMetadata_;
- for (BootstrapMetadata bsmd : bsMetadata)
+ for (StreamRequestMetadata srm : streamRequestMetadata)
{
if (logger_.isDebugEnabled())
- logger_.debug(bsmd.toString());
- Streaming.transferRanges(bsmd.target_, bsmd.ranges_, null);
+ logger_.debug(srm.toString());
+ Streaming.transferRanges(srm.target_, srm.ranges_, null);
}
}
catch (IOException ex)
Propchange: incubator/cassandra/trunk/src/java/org/apache/cassandra/io/StreamRequestVerbHandler.java
------------------------------------------------------------------------------
svn:eol-style = native
Modified: incubator/cassandra/trunk/src/java/org/apache/cassandra/io/Streaming.java
URL: http://svn.apache.org/viewvc/incubator/cassandra/trunk/src/java/org/apache/cassandra/io/Streaming.java?rev=888864&r1=888863&r2=888864&view=diff
==============================================================================
--- incubator/cassandra/trunk/src/java/org/apache/cassandra/io/Streaming.java (original)
+++ incubator/cassandra/trunk/src/java/org/apache/cassandra/io/Streaming.java Wed Dec 9 16:36:31
2009
@@ -126,6 +126,16 @@
}
}
+ /**
+ * Request ranges to be transferred
+ */
+ public static void requestRanges(InetAddress source, Collection<Range> ranges)
+ {
+ StreamRequestMetadata streamRequestMetadata = new StreamRequestMetadata(FBUtilities.getLocalAddress(),
ranges);
+ Message message = StreamRequestMessage.makeStreamRequestMessage(new StreamRequestMessage(streamRequestMetadata));
+ MessagingService.instance().sendOneWay(message, source);
+ }
+
public static class StreamInitiateVerbHandler implements IVerbHandler
{
/*
Modified: incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java
URL: http://svn.apache.org/viewvc/incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java?rev=888864&r1=888863&r2=888864&view=diff
==============================================================================
--- incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java (original)
+++ incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageProxy.java Wed
Dec 9 16:36:31 2009
@@ -41,7 +41,6 @@
import org.apache.cassandra.locator.TokenMetadata;
import org.apache.cassandra.dht.IPartitioner;
import org.apache.cassandra.dht.Range;
-import org.apache.cassandra.dht.Token;
import org.apache.cassandra.gms.FailureDetector;
import org.apache.cassandra.concurrent.StageManager;
@@ -83,14 +82,6 @@
}
};
- private static Comparator<Row> rowComparator = new Comparator<Row>()
- {
- public int compare(Row r1, Row r2)
- {
- return keyComparator.compare(r1.key, r2.key);
- }
- };
-
/**
* Use this method to have this RowMutation applied
* across all replicas. This method will take care
@@ -308,7 +299,6 @@
List<Row> rows = new ArrayList<Row>();
List<IAsyncResult> iars = new ArrayList<IAsyncResult>();
- int commandIndex = 0;
for (ReadCommand command: commands)
{
@@ -337,7 +327,6 @@
ReadResponse response = ReadResponse.serializer().deserialize(bufIn);
if (response.row() != null)
rows.add(response.row());
- commandIndex++;
}
return rows;
}
@@ -412,7 +401,6 @@
for (ReadCommand command: commands)
{
- // TODO: throw a thrift exception if we do not have N nodes
assert !command.isDigestQuery();
ReadCommand readMessageDigestOnly = command.copy();
readMessageDigestOnly.setDigestQuery(true);
@@ -707,7 +695,6 @@
List<InetAddress> endpoints = StorageService.instance().getLiveNaturalEndpoints(command.key);
/* Remove the local storage endpoint from the list. */
endpoints.remove(FBUtilities.getLocalAddress());
- // TODO: throw a thrift exception if we do not have N nodes
if (logger.isDebugEnabled())
logger.debug("weakreadlocal reading " + command);
Modified: incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java
URL: http://svn.apache.org/viewvc/incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java?rev=888864&r1=888863&r2=888864&view=diff
==============================================================================
--- incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java (original)
+++ incubator/cassandra/trunk/src/java/org/apache/cassandra/service/StorageService.java Wed
Dec 9 16:36:31 2009
@@ -42,6 +42,7 @@
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.io.SSTableReader;
import org.apache.cassandra.io.Streaming;
+import org.apache.cassandra.io.StreamRequestVerbHandler;
import org.apache.log4j.Logger;
import org.apache.log4j.Level;
@@ -76,7 +77,7 @@
public final static String streamInitiateDoneVerbHandler_ = "BOOTSTRAP-INITIATE-DONE-VERB-HANDLER";
public final static String streamFinishedVerbHandler_ = "BOOTSTRAP-TERMINATE-VERB-HANDLER";
public final static String dataFileVerbHandler_ = "DATA-FILE-VERB-HANDLER";
- public final static String bootstrapMetadataVerbHandler_ = "BS-METADATA-VERB-HANDLER";
+ public final static String streamRequestVerbHandler_ = "BS-METADATA-VERB-HANDLER";
public final static String rangeVerbHandler_ = "RANGE-VERB-HANDLER";
public final static String rangeSliceVerbHandler_ = "RANGE-SLICE-VERB-HANDLER";
public final static String bootstrapTokenVerbHandler_ = "SPLITS-VERB-HANDLER";
@@ -216,7 +217,7 @@
MessagingService.instance().registerVerbHandlers(rangeSliceVerbHandler_, new RangeSliceVerbHandler());
// see BootStrapper for a summary of how the bootstrap verbs interact
MessagingService.instance().registerVerbHandlers(bootstrapTokenVerbHandler_, new
BootStrapper.BootstrapTokenVerbHandler());
- MessagingService.instance().registerVerbHandlers(bootstrapMetadataVerbHandler_, new
BootstrapMetadataVerbHandler() );
+ MessagingService.instance().registerVerbHandlers(streamRequestVerbHandler_, new StreamRequestVerbHandler()
);
MessagingService.instance().registerVerbHandlers(streamInitiateVerbHandler_, new
Streaming.StreamInitiateVerbHandler());
MessagingService.instance().registerVerbHandlers(streamInitiateDoneVerbHandler_,
new Streaming.StreamInitiateDoneVerbHandler());
MessagingService.instance().registerVerbHandlers(streamFinishedVerbHandler_, new
Streaming.StreamFinishedVerbHandler());
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "Operations" page has been changed by JonathanEllis. http://wiki.apache.org/cassandra/Operations?action=diff&rev1=9&rev2=10 -------------------------------------------------- Note that with !RackAwareStrategy, succeeding nodes along the ring should alternate data centers to avoid hot spots. For instance, if you have nodes A, B, C, and D in increasing Token order, and instead of alternating you place A and B in DC1, and C and D in DC2, then nodes C and A will have disproportionately more data on them because they will be the replica destination for every Token range in the other data center. * The corollary to this is, if you want to start with a single DC and add another later, when you add the second DC you should add as many nodes as you have in the first rather than adding a node or two at a time gradually. - Replication strategy may not be changed without wiping your data and starting over. + Replication strategy is not intended to be changed once live, but if you are sufficiently motivated it can be done with some manual effort: + 1. anticompact each node's primary Range, yielding sstables containing only that Range data + 1. copy those sstables to the nodes responsible for extra replicas under the new strategy + 1. change the strategy and restart == Network topology ==
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "Operations" page has been changed by JonathanEllis. http://wiki.apache.org/cassandra/Operations?action=diff&rev1=8&rev2=9 -------------------------------------------------- * !RackAwareStrategy: replica 2 is is placed in the first node along the ring the belongs in '''another''' data center than the first; the remaining N-2 replicas, if any, are placed on the first nodes along the ring in the '''same''' rack as the first Note that with !RackAwareStrategy, succeeding nodes along the ring should alternate data centers to avoid hot spots. For instance, if you have nodes A, B, C, and D in increasing Token order, and instead of alternating you place A and B in DC1, and C and D in DC2, then nodes C and A will have disproportionately more data on them because they will be the replica destination for every Token range in the other data center. + * The corollary to this is, if you want to start with a single DC and add another later, when you add the second DC you should add as many nodes as you have in the first rather than adding a node or two at a time gradually. Replication strategy may not be changed without wiping your data and starting over.
[ https://issues.apache.org/jira/browse/CASSANDRA-567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-567.
--------------------------------------
Resolution: Fixed
Fix Version/s: (was: 0.9)
0.5
> add querying of supercolumns to commandline, or replace it
> ----------------------------------------------------------
>
> Key: CASSANDRA-567
> URL: https://issues.apache.org/jira/browse/CASSANDRA-567
> Project: Cassandra
> Issue Type: New Feature
> Components: Tools
> Reporter: Jonathan Ellis
> Assignee: Eric Evans
> Priority: Minor
> Fix For: 0.5
>
> Attachments: v1-0001-CASSANDRA-567-refactor-of-executeSet-to-support-super-.txt,
v1-0002-refactor-of-executeGet-to-support-super-columns.txt, v1-0003-cleaner-handling-of-string-encoding-exceptions.txt,
v1-0004-documented-CASSANDRA-567-in-changelog.txt
>
>
> Part of me still wants to rip it out and replace it with http://wiki.apache.org/cassandra/ThriftInterface,
but another part recognizes that getting the thrift compiler working is enough of a bitch
that maybe it is worth going with the Antlr approach if we can make it work.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788135#action_12788135
]
Jonathan Ellis commented on CASSANDRA-617:
------------------------------------------
Wouldn't it be worth making gossip state names an enum? This would (a) make it harder to
screw things up with name collisions, and (b) make name part of gossip load almost negligible.
Avinash also suggested:
* Compress the messages which are being sent. I have seen 5:1 compression is achievable.
So if you send 117 per the JIRA this is automatically 500+.
* If and if absolutely necessary move to TCP and compress and send all digests. Then all
the problems go away.
IMO [jbellis again] if we can get to ~500 with compression and the simpler fixes (your Syn
fix, move state optimization, enum names) then that gives us a long time before any current
user of Cassandra needs to worry about the complexity of adding multiple messages per round
(or the even bigger change of moving to TCP).
> gossiper scalability
> --------------------
>
> Key: CASSANDRA-617
> URL: https://issues.apache.org/jira/browse/CASSANDRA-617
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.5
> Reporter: Jaakko Laine
> Fix For: 0.5
>
> Attachments: averages.png, spread_100.png, spread_50.png, spread_90.png
>
>
> Improve gossiper scalability.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788115#action_12788115
]
Jaakko Laine commented on CASSANDRA-617:
----------------------------------------
As for the fixes...
(1) I think the easiest way would be to give Gossiper a chance to send multiple Syn messages
per round. If it cannot send "enough" (enough being certain threshold percentage) digests
compared to how many endpoints there are in all, it could send multiple Syn messages to the
same host. Gossiper messages are independent of each other, so they can be handled separately.
The downside would be that there is potentially quite a number of endpoint states to handle
per one round. However, since current way is for all nodes to know about all other nodes,
this will put a cap to scalability at some point in any case. (No idea at what point, though).
(2a) Depends on (1). If we can send multiple messages per round, then this one goes away.
(2b) Multiple messages will also solve this. However, if we stick to one message per round,
we might consider guaranteeing certain chunk of packet for both purposes.
(3a) When handling Syn message with only one digest (the sending node), include random endpoint
state information in the reply as much as there is room. This way a new node does not need
to wait for syn message from the seed to start gathering information about other nodes.
(3b) Might not be a big issue, but in order to squeeze more endpoint states to gossip messages,
perhaps it would be good idea to reduce the number of "move" states from four separate states
to one. This would decrease the total amount of data in Gossiper "cloud" potentially by tens
of percents. Would speed up the process of new node gathering node info.
> gossiper scalability
> --------------------
>
> Key: CASSANDRA-617
> URL: https://issues.apache.org/jira/browse/CASSANDRA-617
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.5
> Reporter: Jaakko Laine
> Fix For: 0.5
>
> Attachments: averages.png, spread_100.png, spread_50.png, spread_90.png
>
>
> Improve gossiper scalability.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jaakko Laine updated CASSANDRA-617:
-----------------------------------
Attachment: spread_100.png
> gossiper scalability
> --------------------
>
> Key: CASSANDRA-617
> URL: https://issues.apache.org/jira/browse/CASSANDRA-617
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.5
> Reporter: Jaakko Laine
> Fix For: 0.5
>
> Attachments: averages.png, spread_100.png, spread_50.png, spread_90.png
>
>
> Improve gossiper scalability.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jaakko Laine updated CASSANDRA-617:
-----------------------------------
Attachment: spread_90.png
spread_50.png
averages.png
I've run some simulated gossiper tests and there are some issues with Gossiper scalability:
(1) Overall capacity to spread state information:
Due to gossiper packet size, at most 117 gossip digest messages fit into one GossipDigestSynMessage.
This means that if there are more than 117 nodes, at one round only a subset of all digests
(endpoint states) can be gossiped. This can cause starvation as some nodes' state information
has to potentially queue for long time before getting a chance to spread (for example if there
are 234 nodes, there's only 50% chance for state information to spread at each stage). Due
to this, the time it takes for one state information to reach all nodes, grows logarithmically
only to 117 nodes. Growth rate after this seems to be linearish (it is not exponential as
randomness and multiple paths between any two nodes dampen the effect of worsening max_digest_size
/ all_endpoints_size ratio). Attached chart (averages.png) shows average amount of rounds
it takes for a piece of gossip to reach given percentage of nodes. As can be seen, after cluster
size exceeds 117 nodes, the curves take a sharp turn upwards.
As can also be seen, when cluster size grows, 98% curve stays considerably lower than 100%.
When there are only a few nodes without certain piece of gossip, max_digest_size / all_endpoints_size
ratio plays bigger role and it is more difficult for certain state information to reach the
last nodes.
Attached there are also spread_100.png, spread_90.png and spread_50.png charts. These show
minimum, maximum and average time to reach 100%, 90% and 50% of the nodes respectively. Samples
per one stage (number of nodes) were only 12 gossips, so there is some room for error, but
should give some indication as to how the gossiper behaves in larger clusters.
(2) Ability of Ack message to spread endpoint state information
(a) Size of GossipDigestAckMessage is similarly restricted, so the amount of gossip digests
and endpoint states it can carry is also limited. In the simulation, once cluster size reaches
about 80-90 nodes, not all endpoint states can be included in Ack message. This means that
on top of delays caused by Syn message limitations, also Ack message capacity will cause delays
in endpoint state propagation.
(b) Another related issue is how GossipDigestAckMessage size is divided between digests and
endpoint states. Currently all digests are included first, and then, if there is room, as
many endpoint states will be included as will fit. In the simulation this was not a problem,
but it might happen that digests take so much room that not many endpoint states can be included.
(3) New nodes entering a large cluster
(a) When a new node enters a ring, it will first gossip to a seed and let it know that it
has joined the cluster. However, the way SYN/ACK/ACK2 exchange works, the seed will not volunteer
information about any other nodes it knows about. Only when a seed randomly chooses this node
to gossip to, the new node will know (through the arriving Syn message) about other nodes.
In a large cluster, this chance might be very small (less than a percent), so the node might
need to wait for considerable time before getting knowledge of other nodes.
(b) Another related issue is endpoint state size. There are currently four "move" application
states, which make the whole endpoint state rather big. During normal operation these states
are of course delivered as deltas, but when a new node enters the ring, full state needs to
be delivered. Only 9-10 of these states fit to one Ack message, so it will take some time
before all data is delivered in a big cluster.
> gossiper scalability
> --------------------
>
> Key: CASSANDRA-617
> URL: https://issues.apache.org/jira/browse/CASSANDRA-617
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.5
> Reporter: Jaakko Laine
> Fix For: 0.5
>
> Attachments: averages.png, spread_100.png, spread_50.png, spread_90.png
>
>
> Improve gossiper scalability.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
gossiper scalability
--------------------
Key: CASSANDRA-617
URL: https://issues.apache.org/jira/browse/CASSANDRA-617
Project: Cassandra
Issue Type: Improvement
Components: Core
Affects Versions: 0.5
Reporter: Jaakko Laine
Fix For: 0.5
Improve gossiper scalability.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification.
The "ClientExamples" page has been changed by gdusbabek.
The comment on this change is: Indicated that the fat client is not a part of 0.5..
http://wiki.apache.org/cassandra/ClientExamples?action=diff&rev1=41&rev2=42
--------------------------------------------------
}
}}}
- Alternatively, there is a Java "Fat Client" that can be used to bring up a node in client-only
mode. A client node may participate in reads or writes and has the added benefit of avoid
thrift-related overhead. The following example comes from /contrib/client_only:
+ In the trunk, there is a Java "Fat Client" that can be used to bring up a node in client-only
mode. A client node may participate in reads or writes and has the added benefit of avoid
thrift-related overhead. This is due to be released '''after''' version 0.5 (likely 0.9).
+
+ The following example comes from /contrib/client_only:
Writing
{{{
[ https://issues.apache.org/jira/browse/CASSANDRA-564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jaakko Laine updated CASSANDRA-564:
-----------------------------------
Attachment: 564-2-add-removetoken-command.patch
564-1-replace-bootstrapmetadata-with-streamrequest.patch
New Version
564-1
- Rename BootstrapMetadata and friends to StreamRequest* as that is what those essentially
are. Move them to cassandra.io where rest of streaming stuff resides.
- Remove some unneeded imports from BootStrapper
564-2
Add removeToken command and related stuff to StorageService. This now uses STATE_LEFT state,
but we might also create another state for this, if needed.
> Provide recoverability when a node dies and it is impossible to get the same IP.
> --------------------------------------------------------------------------------
>
> Key: CASSANDRA-564
> URL: https://issues.apache.org/jira/browse/CASSANDRA-564
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Anthony Molinaro
> Priority: Minor
> Fix For: 0.5
>
> Attachments: 564-1-replace-bootstrapmetadata-with-streamrequest.patch, 564-2-add-removetoken-command.patch
>
>
> From the descriptions on the mailing list, when a node dies permanently from hardware
or other failure and you need to replace it, it must have the same IP. For people running
in cloud environments, this is often times impossible. So it would be very useful if there
was a way to replace a node with a new node without requiring the same IP.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788067#action_12788067
]
Hudson commented on CASSANDRA-616:
----------------------------------
Integrated in Cassandra #282 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/282/])
avoid anti-entropy checks on the System table. patch by Stu Hood; reviewed by jbellis
for
> Anti-entropy might attempt to repair the system table
> -----------------------------------------------------
>
> Key: CASSANDRA-616
> URL: https://issues.apache.org/jira/browse/CASSANDRA-616
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 0.5
> Reporter: Stu Hood
> Priority: Blocker
> Fix For: 0.5
>
> Attachments: 616-omg-aes.diff
>
>
> It is currently possible for AntiEntropyService to decide to 'repair' the system table,
which would cause pandemonium, with one node losing its stored token.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788072#action_12788072
]
Hudson commented on CASSANDRA-608:
----------------------------------
Integrated in Cassandra #282 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/282/])
r/m unused code. patch by Stu Hood; reviewed by jbellis for
> r/m dead code from MerkleTree, AntiEntropyService
> -------------------------------------------------
>
> Key: CASSANDRA-608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-608
> Project: Cassandra
> Issue Type: Task
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 608-dead-code.diff, 608-dead-code.diff
>
>
> method parameters in MerkleTree are the worst offenders
> http://wiki.apache.org/cassandra/RunningCassandraInIDEA describes how to set up Cassandra
in IDEA; the OSS build from http://jetbrains.org/ is fine. alternatively, Eric assures me
that Eclipse can also identify dead code easily but we don't have a wiki page for that. :)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788065#action_12788065
]
Hudson commented on CASSANDRA-324:
----------------------------------
Integrated in Cassandra #282 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/282/])
add documentation to thrift interface
Patch by Gary Dusbabek; reviewed by eevans for
> Add documentation to Thrift interface
> -------------------------------------
>
> Key: CASSANDRA-324
> URL: https://issues.apache.org/jira/browse/CASSANDRA-324
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation & website
> Reporter: Michael Greene
> Assignee: Gary Dusbabek
> Priority: Minor
> Fix For: 0.5
>
> Attachments: 324-v1.patch, 324-v2.diff, 324-v3-no-html.patch, v1-0001-CASSANDRA-324-build-javadocs-for-thrift-generated-code.txt
>
>
> We should be using the doctext feature of Thrift to document all of our enums, structs,
and services.
> Rather than leaving individual wrapper libraries to document the interface, it should
be consolidated into one place (the .thrift file). It can then be used to generate javadocs
for the stuff that gets placed in gen-java, documentation for the Thrift-generated client
libraries, or generate nifty html using Thrift's own --gen html.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788073#action_12788073
]
Hudson commented on CASSANDRA-520:
----------------------------------
Integrated in Cassandra #282 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/282/])
> Implement Range Repairs
> -----------------------
>
> Key: CASSANDRA-520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-520
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.5
>
> Attachments: 520-bulk-1-use-streaming-api.diff, 520-bulk-1-use-streaming-api.diff,
520-bulk-1-use-streaming-api.diff, 520-bulk-2-repair-strategy-by-fraction.diff, 520-bulk-2-repair-strategy-by-fraction.diff,
520-bulk-2-repair-strategy-by-fraction.diff
>
>
> After CASSANDRA-193, the TreeRequest/Response conversation will have generated a full
list of Ranges that disagree between nodes. We need an operation that can efficiently batch
repair the ranges, similar to what happens during read repair for a single key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[ https://issues.apache.org/jira/browse/CASSANDRA-604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788069#action_12788069
]
Hudson commented on CASSANDRA-604:
----------------------------------
Integrated in Cassandra #282 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/282/])
only GC when compacting all sstables, to avoid situations where the data a tombstone is
intended to supress is in an sstable that is not part of the compaction set.
patch by jbellis; reviewed by Ramzi Rabah for
> Compactions might remove tombstones without removing the actual data
> --------------------------------------------------------------------
>
> Key: CASSANDRA-604
> URL: https://issues.apache.org/jira/browse/CASSANDRA-604
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cent-OS
> Reporter: Ramzi Rabah
> Assignee: Jonathan Ellis
> Fix For: 0.5
>
> Attachments: 604.patch
>
>
> I was looking at the code for compaction, and noticed that when we are doing compactions
during the normal course of
> Cassandra, we call:
> for (List<SSTableReader> sstables :
> getCompactionBuckets(ssTables_, 50L * 1024L * 1024L))
> {
> if (sstables.size() < minThreshold)
> {
> continue;
> }
> other wise docompactions...
> where getCompactionBuckets puts in buckets very small files, or files
> that are 0.5-1.5 of each other's sizes. It will only compact those if
> they are >= minimum threshold which is 4 by default.
> So far so good. Now how about this scenario, I have an old entry that
> I inserted long time ago and that was compacted into a 75MB file.
> There are fewer 75MB files than 4. I do many deletes, and I end with 4
> extra sstable files filled with tombstones, each about 300 MB large.
> These 4 files are compacted together and in the compaction code, if
> the tombstone is there we don't copy it over to the new file. Now
> since we did not compact the 75MB files, but we compacted the
> tombstone files, that leaves us with the tombstone gone, but
> the data still intact in the 75MB file. If we compacted all the
> files together I don't think that would be a problem, but since we
> only compact 4, this potentially leaves data not cleaned.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.