> nodetool compactionstats, gives > > pending tasks: 13120 If there are no errors in the log, I would say this is a bug. Cheers ----------------- Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 19/06/2013, at 11:41 AM, Franc Carter <franc.carter@sirca.org.au> wrote: > On Wed, Jun 19, 2013 at 9:34 AM, Bryan Talbot <btalbot@aeriagames.com> wrote: > Manual compaction for LCS doesn't really do much. It certainly doesn't compact all those little files into bigger files. What makes you think that compactions are not occurring? > > Yeah, that's what I thought, however:- > > nodetool compactionstats, gives > > pending tasks: 13120 > Active compaction remaining time : n/a > > when I run nodetool compact in a loop the pending tasks goes down gradually. > > This node also has vastly higher latencies (x10) than the other nodes. I saw this with a previous CF than I 'manually compacted', and when the pending tasks reached low numbers (stuck on 9) then latencies were back to low milliseconds > > cheers > > -Bryan > > > > On Tue, Jun 18, 2013 at 3:59 PM, Franc Carter <franc.carter@sirca.org.au> wrote: > On Sat, Jun 15, 2013 at 11:49 AM, Franc Carter <franc.carter@sirca.org.au> wrote: > On Sat, Jun 15, 2013 at 8:48 AM, Robert Coli <rcoli@eventbrite.com> wrote: > On Wed, Jun 12, 2013 at 3:26 PM, Franc Carter <franc.carter@sirca.org.au> wrote: > > We are running a test system with Leveled compaction on Cassandra-1.2.4. > > While doing an initial load of the data one of the nodes ran out of file > > descriptors and since then it hasn't been automatically compacting. > > You have (at least) two options : > > 1) increase file descriptors available to Cassandra with ulimit, if possible > 2) increase the size of your sstables with levelled compaction, such > that you have fewer of them > > Oops, I wasn't clear enough. > > I have increased the number of file descriptors and no longer have a file descriptor issue. However the node still doesn't compact automatically. If I run a 'nodetool compact' it will do a small amount of compaction and then stop. The Column Family is using LCS > > Any ideas on this - compaction is still not automatically running for one of my nodes > > thanks > > > cheers > > > =Rob > > > > -- > Franc Carter | Systems architect | Sirca Ltd > franc.carter@sirca.org.au | www.sirca.org.au > Tel: +61 2 8355 2514 > Level 4, 55 Harrington St, The Rocks NSW 2000 > PO Box H58, Australia Square, Sydney NSW 1215 > > > > > -- > Franc Carter | Systems architect | Sirca Ltd > franc.carter@sirca.org.au | www.sirca.org.au > Tel: +61 2 8355 2514 > Level 4, 55 Harrington St, The Rocks NSW 2000 > PO Box H58, Australia Square, Sydney NSW 1215 > > > > > > -- > Franc Carter | Systems architect | Sirca Ltd > franc.carter@sirca.org.au | www.sirca.org.au > Tel: +61 2 8355 2514 > Level 4, 55 Harrington St, The Rocks NSW 2000 > PO Box H58, Australia Square, Sydney NSW 1215 >
> 1) Is there any implication in running nodetool repair immediately after bringing a new node up (before key migration process is completed) ? > Will it cause some race conditions ? Or will it result in some part of the space never be reclaimed ? Repair will only be concerned with data that the node is replica for. And cleanup is only concerned with data that the node is no longer a replica for. AFAIK they should be able to run concurrently, but I would avoid it incase there are some edge cases. > 2) How can I figure out the status of key migration in Cassandra? Not sure what you mean by key migration. If you are talking about a token move or node bootstrap you can get some idea from nodetool compactionstats and nodetool netstats. FWIW I think nodetool cleanup is less aggressive than repair. Repair reads all the data and creates a hash, cleanup just reads it from one file and writes a new file dropping rows that no longer belong. It's probably uses less CPU than compaction as it does not merge row fragments. Cheers ----------------- Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 19/06/2013, at 10:48 AM, Emalayan Vairavanathan <svemalayan@yahoo.com> wrote: > Thank you all. > > I have two more question. > > 1) Is there any implication in running nodetool repair immediately after bringing a new node up (before key migration process is completed) ? > Will it cause some race conditions ? Or will it result in some part of the space never be reclaimed ? > > 2) How can I figure out the status of key migration in Cassandra? > > Thank you > Emalayan > > From: Richard Low <richard@wentnet.com> > To: user@cassandra.apache.org; Emalayan Vairavanathan <svemalayan@yahoo.com> > Sent: Tuesday, 18 June 2013 12:11 AM > Subject: Re: [Cassandra] Expanding a Cassandra cluster > > On 10 June 2013 22:00, Emalayan Vairavanathan <svemalayan@yahoo.com> wrote: > > b) Will Cassandra automatically take care of removing obsolete keys in future ? > > In a future version Cassandra should automatically clean up for you: > > https://issues.apache.org/jira/browse/CASSANDRA-5051 > > Right now though you have to run cleanup eventually or the space will never be reclaimed. > > Richard. > >
> As for the thrift side (i.e. using Hector or Astyanax), anyone have a crafty way to inject something? The only thing I've ever heard of coming close was a thrift bug that allowed a malformed request to crash the server. But that was a while ago https://issues.apache.org/jira/browse/CASSANDRA-475 Cheers ----------------- Aaron Morton Freelance Cassandra Consultant New Zealand @aaronmorton http://www.thelastpickle.com On 19/06/2013, at 1:46 AM, Brian O'Neill <bone@alumni.brown.edu> wrote: > > Perfect. Thanks Sylvain. That is exactly the input I was looking for, and I agree completely. > (t's easy enough to protect against) > > As for the thrift side (i.e. using Hector or Astyanax), anyone have a crafty way to inject something? > > At first glance, it doesn't appear possible, but I'm not 100% confident making that assertion. > > -brian > > --- > Brian O'Neill > Lead Architect, Software Development > Health Market Science > The Science of Better Results > 2700 Horizon Drive • King of Prussia, PA • 19406 > M: 215.588.6024 • @boneill42 • > healthmarketscience.com > > This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. > > > > From: Sylvain Lebresne <sylvain@datastax.com> > Reply-To: <user@cassandra.apache.org> > Date: Tuesday, June 18, 2013 8:51 AM > To: "user@cassandra.apache.org" <user@cassandra.apache.org> > Subject: Re: "SQL" Injection C* (via CQL & Thrift) > > If you're not careful, then "CQL injection" is possible. > > Say you naively build you query with > "UPDATE foo SET col='" + user_input + "' WHERE key = 'k'" > then if user_input is "foo' AND col2='bar", your user will have overwritten a column it shouldn't have been able to. And something equivalent in a BATCH statement could allow to overwrite/delete some random row in some random table. > > Now CQL being much more restricted than SQL (no subqueries, no generic transaction, ...), the extent of what you can do with a CQL injection is way smaller than in SQL. But you do have to be careful. > > As far as the Datastax java driver is concerned, you can fairly easily protect yourself by using either: > 1) prepared statements: if the user input is a prepared variable, there is nothing the user can do (it's "equivalent" to the thrift situation). > 2) using the query builder: it will escape quotes in the strings you provided, thuse avoiding injection. > > So I would say that injections are definitively possible if you concatenate strings too naively, but I don't think preventing them is very hard. > > -- > Sylvain > > > On Tue, Jun 18, 2013 at 2:02 PM, Brian O'Neill <bone@alumni.brown.edu> wrote: >> >> Mostly for fun, I wanted to throw this out there... >> >> We are undergoing a security audit for our platform (C* + Elastic Search + Storm). One component of that audit is susceptibility to SQL injection. I was wondering if anyone has attempted to construct a SQL injection attack against Cassandra? Is it even possible? >> >> I know the code paths fairly well, but... >> Does there exists a path in the code whereby user data gets interpreted, which could be exploited to perform user operations? >> >> From the Thrift side of things, I've always felt safe. Data is opaque. Serializers are used to convert it to Bytes, and C* doesn't ever really do anything with the data. >> >> In examining the CQL java-driver, it looks like there might be a bit more exposure to injection. (or even CQL over Thrift) I haven't dug into the code yet, but dependent on which flavor of the API you are using, you may be including user data in your statements. >> >> Does anyone know if the CQL java-driver does anything to protect against injection? Or is it possible to say that the syntax is strict enough that any embedded operations in data would not parse? >> >> just some food for thought... >> I'll be digging into this over the next couple weeks. If people are interested, I can throw a blog post out there with the findings. >> >> -brian >> >> -- >> Brian ONeill >> Lead Architect, Health Market Science (http://healthmarketscience.com) >> mobile:215.588.6024 >> blog: http://brianoneill.blogspot.com/ >> twitter: @boneill42 >
Thanks Serge Simon On Thu, Jun 20, 2013 at 10:48 AM, Serge Fonville <serge.fonville@gmail.com> wrote: > Also, after a quick Google. > > http://wiki.apache.org/cassandra/CassandraLimitations states values cannot > exceed 2GB, it also answers you offset question > > HTH > Kind regards/met vriendelijke groet, > > Serge Fonville > > http://www.sergefonville.nl > > Convince Microsoft! > They need to add TRUNCATE PARTITION in SQL Server > https://connect.microsoft.com/SQLServer/feedback/details/417926/truncate-partition-of-partitioned-table > > > 2013/6/20 Sachin Sinha <sinha.sachin@gmail.com> >> >> Fragment them in rows, that will help. >> >> >> On 20 June 2013 09:43, Simon Majou <simon@majou.org> wrote: >>> >>> Hello, >>> >>> If I store a video into a column, how can I get a fragment of it >>> without having to download it entirely ? Is there a way to give an >>> offset on a column ? >>> >>> Do I have to fragment it over a lot of small fixed sizes columns ? Is >>> there any disadvantage to do so ? For example fragment a 10GB file >>> into 1 000 columns of 10 MB ? >>> >>> Simon >> >> >
Also, after a quick Google. http://wiki.apache.org/cassandra/CassandraLimitations states values cannot exceed 2GB, it also answers you offset question HTH Kind regards/met vriendelijke groet, Serge Fonville http://www.sergefonville.nl Convince Microsoft! They need to add TRUNCATE PARTITION in SQL Server https://connect.microsoft.com/SQLServer/feedback/details/417926/truncate-partition-of-partitioned-table 2013/6/20 Sachin Sinha <sinha.sachin@gmail.com> > Fragment them in rows, that will help. > > > On 20 June 2013 09:43, Simon Majou <simon@majou.org> wrote: > >> Hello, >> >> If I store a video into a column, how can I get a fragment of it >> without having to download it entirely ? Is there a way to give an >> offset on a column ? >> >> Do I have to fragment it over a lot of small fixed sizes columns ? Is >> there any disadvantage to do so ? For example fragment a 10GB file >> into 1 000 columns of 10 MB ? >> >> Simon >> > >
Fragment them in rows, that will help. On 20 June 2013 09:43, Simon Majou <simon@majou.org> wrote: > Hello, > > If I store a video into a column, how can I get a fragment of it > without having to download it entirely ? Is there a way to give an > offset on a column ? > > Do I have to fragment it over a lot of small fixed sizes columns ? Is > there any disadvantage to do so ? For example fragment a 10GB file > into 1 000 columns of 10 MB ? > > Simon >
Hello, If I store a video into a column, how can I get a fragment of it without having to download it entirely ? Is there a way to give an offset on a column ? Do I have to fragment it over a lot of small fixed sizes columns ? Is there any disadvantage to do so ? For example fragment a 10GB file into 1 000 columns of 10 MB ? Simon
12.3 GB data per node (only one ńode). 16GB RAM. In virtual environment with the CPU specified as "8 cores", average CPU use is close to 0% (basically no load, around 12 requests / sec, mostly from OpsCenter). Average memory use is 4GB. Around 1GB heap used by Cassandra (out of 4GB). 2013/6/19 Mohit Anchlia <mohitanchlia@gmail.com> > How much data do you have per node? > How much RAM per node? > How much CPU per node? > What is the avg CPU and memory usage? > > On Wed, Jun 19, 2013 at 12:16 AM, Joel Samuelsson < > samuelsson.joel@gmail.com> wrote: > >> My Cassandra ps info: >> >> root 26791 1 0 07:14 ? 00:00:00 /usr/bin/jsvc -user >> cassandra -home /opt/java/64/jre1.6.0_32/bin/../ -pidfile >> /var/run/cassandra.pid -errfile &1 -outfile /var/log/cassandra/output.log >> -cp >> /usr/share/cassandra/lib/antlr-3.2.jar:/usr/share/cassandra/lib/avro-1.4.0-fixes.jar:/usr/share/cassandra/lib/avro-1.4.0-sources-fixes.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang-2.6.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.3.jar:/usr/share/cassandra/lib/guava-13.0.1.jar:/usr/share/cassandra/lib/high-scale-lib-1.1.2.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jline-1.0.jar:/usr/share/cassandra/lib/jna.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.7.0.jar:/usr/share/cassandra/lib/log4j-1.2.16.jar:/usr/share/cassandra/lib/lz4-1.1.0.jar:/usr/share/cassandra/lib/metrics-core-2.0.3.jar:/usr/share/cassandra/lib/netty-3.5.9.Final.jar:/usr/share/cassandra/lib/servlet-api-2.5-20081211.jar:/usr/share/cassandra/lib/slf4j-api-1.7.2.jar:/usr/share/cassandra/lib/slf4j-log4j12-1.7.2.jar:/usr/share/cassandra/lib/snakeyaml-1.6.jar:/usr/share/cassandra/lib/snappy-java-1.0.4.1.jar:/usr/share/cassandra/lib/snaptree-0.1.jar:/usr/share/cassandra/apache-cassandra-1.2.5.jar:/usr/share/cassandra/apache-cassandra-thrift-1.2.5.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar:/usr/share/java/jna.jar:/etc/cassandra:/usr/share/java/commons-daemon.jar >> -Dlog4j.configuration=log4j-server.properties >> -Dlog4j.defaultInitOverride=true >> -XX:HeapDumpPath=/var/lib/cassandra/java_1371626058.hprof >> -XX:ErrorFile=/var/lib/cassandra/hs_err_1371626058.log -ea >> -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities >> -XX:ThreadPriorityPolicy=42 -Xms4004M -Xmx4004M -Xmn800M >> -XX:+HeapDumpOnOutOfMemoryError -Xss180k -XX:+UseParNewGC >> -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 >> -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 >> -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseTLAB >> -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.port=7199 >> -Dcom.sun.management.jmxremote.ssl=false >> -Dcom.sun.management.jmxremote.authenticate=false >> org.apache.cassandra.service.CassandraDaemon >> 103 26792 26791 99 07:14 ? 854015-22:02:22 /usr/bin/jsvc >> -user cassandra -home /opt/java/64/jre1.6.0_32/bin/../ -pidfile >> /var/run/cassandra.pid -errfile &1 -outfile /var/log/cassandra/output.log >> -cp >> /usr/share/cassandra/lib/antlr-3.2.jar:/usr/share/cassandra/lib/avro-1.4.0-fixes.jar:/usr/share/cassandra/lib/avro-1.4.0-sources-fixes.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang-2.6.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.3.jar:/usr/share/cassandra/lib/guava-13.0.1.jar:/usr/share/cassandra/lib/high-scale-lib-1.1.2.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jline-1.0.jar:/usr/share/cassandra/lib/jna.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.7.0.jar:/usr/share/cassandra/lib/log4j-1.2.16.jar:/usr/share/cassandra/lib/lz4-1.1.0.jar:/usr/share/cassandra/lib/metrics-core-2.0.3.jar:/usr/share/cassandra/lib/netty-3.5.9.Final.jar:/usr/share/cassandra/lib/servlet-api-2.5-20081211.jar:/usr/share/cassandra/lib/slf4j-api-1.7.2.jar:/usr/share/cassandra/lib/slf4j-log4j12-1.7.2.jar:/usr/share/cassandra/lib/snakeyaml-1.6.jar:/usr/share/cassandra/lib/snappy-java-1.0.4.1.jar:/usr/share/cassandra/lib/snaptree-0.1.jar:/usr/share/cassandra/apache-cassandra-1.2.5.jar:/usr/share/cassandra/apache-cassandra-thrift-1.2.5.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar:/usr/share/java/jna.jar:/etc/cassandra:/usr/share/java/commons-daemon.jar >> -Dlog4j.configuration=log4j-server.properties >> -Dlog4j.defaultInitOverride=true >> -XX:HeapDumpPath=/var/lib/cassandra/java_1371626058.hprof >> -XX:ErrorFile=/var/lib/cassandra/hs_err_1371626058.log -ea >> -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities >> -XX:ThreadPriorityPolicy=42 -Xms4004M -Xmx4004M -Xmn800M >> -XX:+HeapDumpOnOutOfMemoryError -Xss180k -XX:+UseParNewGC >> -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 >> -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 >> -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseTLAB >> -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.port=7199 >> -Dcom.sun.management.jmxremote.ssl=false >> -Dcom.sun.management.jmxremote.authenticate=false >> org.apache.cassandra.service.CassandraDaemon >> >> Is it normal to have two processes like this? >> >> >
Hi All, 1) What will happen if I run nodetool cleanup immediately after bringing a new node up (i.e. before the key migration process is completed) ? Will it cause some race conditions ? Or will it result in some part of the space never be reclaimed ? 2) After adding a new machine, how can I make sure that the key migration is completed ? Should I run nodetool netstats on all the nodes ? Is there any better way ? Thank you Emalayan
Thanks Edward, Ben and Dean for the pointers. Yes, I am using Java and these sounds promising for unit testing, at least. Regards, Shahab On Wed, Jun 19, 2013 at 9:58 AM, Edward Capriolo <edlinuxguru@gmail.com>wrote: > You really do not need much in java you can use the embedded server. > Hector wrap a simple class around thiscalled EmbeddedServerHelper > > > On Wednesday, June 19, 2013, Ben Boule <Ben_Boule@rapid7.com> wrote: > > Hi Shabab, > > > > Cassandra-Unit has been helpful for us for running unit tests without > requiring a real cassandra instance to be running. We only use this to > test our "DAO" code which interacts with the Cassandra client. It > basically starts up an embedded instance of cassandra and fools your > client/driver into using it. It uses a non-standard port and you just need > to make sure you can set the port as a parameter into your client code. > > > > https://github.com/jsevellec/cassandra-unit > > > > One important thing is to either clear out the keyspace in between tests > or carefully separate your data so different tests don't collide with each > other in the embedded database. > > > > Setup/tear down time is pretty reasonable. > > > > Ben > > ________________________________ > > From: Shahab Yunus [shahab.yunus@gmail.com] > > Sent: Wednesday, June 19, 2013 8:46 AM > > To: user@cassandra.apache.org > > Subject: Re: Unit Testing Cassandra > > > > Thanks Stephen for you reply and explanation. My bad that I mixed those > up and wasn't clear enough. Yes, I have different 2 requests/questions. > > 1) One is for the unit testing. > > 2) Second (in which I am more interested in) is for performance > (stress/load) testing. Let us keep integration aside for now. > > I do see some stuff out there but wanted to know recommendations from > the community given their experience. > > Regards, > > Shahab > > > > On Wed, Jun 19, 2013 at 3:15 AM, Stephen Connolly < > stephen.alan.connolly@gmail.com> wrote: > >> > >> Unit testing means testing in isolation the smallest part. > >> Unit tests should not take more than a few milliseconds to set up and > verify their assertions. > >> As such, if your code is not factored well for testing, you would > typically use mocking (either by hand, or with mocking libraries) to mock > out the bits not under test. > >> Extensive use of mocks is usually a smell of code that is not well > designed *for testing* > >> If you intend to test components integrated together... That is > integration testing. > >> If you intend to test performance of the whole or significant parts of > the whole... That is performance testing. > >> When searching for the above, you will not get much luck if you are > looking for them in the context of "unit testing" as those things are > *outside the scope of unit testing" > >> > >> On Wednesday, 19 June 2013, Shahab Yunus wrote: > >>> > >>> Hello, > >>> > >>> Can anyone suggest a good/popular Unit Test tools/frameworks/utilities > out > >>> there for unit testing Cassandra stores? I am looking for testing from > performance/load and monitoring perspective. I am using 1.2. > >>> > >>> Thanks a lot. > >>> > >>> Regards, > >>> Shahab > >> > >> > >> -- > >> Sent from my phone > > > > This electronic message contains information which may be confidential > or privileged. The information is intended for the use of the individual or > entity named above. If you are not the intended recipient, be aware that > any disclosure, copying, distribution or use of the contents of this > information is prohibited. If you have received this electronic > transmission in error, please notify us by e-mail at ( > postmaster@rapid7.com) immediately. >
Rob, Thanks. I was not aware of that. So we can avoid repair if there is no hardware failure...I found a blog: http://www.datastax.com/dev/blog/modern-hinted-handoff -Wei ----- Original Message ----- From: "Robert Coli" <rcoli@eventbrite.com> To: user@cassandra.apache.org, "Wei Zhu" <wz1975@yahoo.com> Sent: Wednesday, June 19, 2013 12:58:45 PM Subject: Re: Data not fully replicated with 2 nodes and replication factor 2 On Wed, Jun 19, 2013 at 11:43 AM, Wei Zhu <wz1975@yahoo.com> wrote: > I think hints are only stored when the other node is down, not on the > dropped mutations. (Correct me if I am wrong, actually it's not a bad idea > to store hints for dropped mutations and replay them later?) This used to be the way it worked pre-1.0... https://issues.apache.org/jira/browse/CASSANDRA-2034 In modern cassandra, anything but a successful ack from a coordinated write results in a hint on the coordinator. > To solve your issue, as I mentioned, either do nodetool repair, or increase > your consistency level. By the way, you probably write faster than your > cluster can handle if you see that many dropped mutations. If his hints are ultimately delivered, OP should not "need" repair to be consistent. =Rob
On Thu, Jun 20, 2013 at 9:18 AM, Raihan Jamal <jamalraihan@gmail.com> wrote: > I am trying to see whether there will be any performance difference > between Cassandra 1.0.8 vs Cassandra 1.2.2 for reading the data mainly? > > Has anyone seen any major performance difference? > We are part way through a performance comparison between 1.0.9 with Size Tiered Compaction and 1.2.4 with Leveled Compaction - for our use case it looks like a significant performance improvement on the read side. We are finding compaction lags when we do very large bulk loads, but for us this is an initialisation task and that's a reasonable trade-off cheers -- *Franc Carter* | Systems architect | Sirca Ltd <marc.zianideferranti@sirca.org.au> franc.carter@sirca.org.au | www.sirca.org.au Tel: +61 2 8355 2514 Level 4, 55 Harrington St, The Rocks NSW 2000 PO Box H58, Australia Square, Sydney NSW 1215
I am trying to see whether there will be any performance difference between Cassandra 1.0.8 vs Cassandra 1.2.2 for reading the data mainly? Has anyone seen any major performance difference?
Hi, I couldn't find any information on the following error so I apologize if it has already been discussed. On some of my nodes, I'm getting the following exception when cassandra starts up: 2013-06-19 22:17:39.480414500 Exception encountered during startup: unable to find sufficient sources for streaming range (-4250921392403750427,-4250887922781325324] 2013-06-19 22:17:39.482733500 ERROR Exception in thread Thread[StorageServiceShutdownHook,5,main] (CassandraDaemon.java:org.apache.cassandra.service.CassandraDaemon$1:175) 2013-06-19 22:17:39.482735500 java.lang.NullPointerException 2013-06-19 22:17:39.482735500 at org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:321) 2013-06-19 22:17:39.482736500 at org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:362) 2013-06-19 22:17:39.482736500 at org.apache.cassandra.service.StorageService.access$000(StorageService.java:88) 2013-06-19 22:17:39.482751500 at org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:513) 2013-06-19 22:17:39.482752500 at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 2013-06-19 22:17:39.482752500 at java.lang.Thread.run(Thread.java:662) Can someone point me to more information about what could cause this error? Faraaz
On Wed, Jun 19, 2013 at 11:43 AM, Wei Zhu <wz1975@yahoo.com> wrote: > I think hints are only stored when the other node is down, not on the > dropped mutations. (Correct me if I am wrong, actually it's not a bad idea > to store hints for dropped mutations and replay them later?) This used to be the way it worked pre-1.0... https://issues.apache.org/jira/browse/CASSANDRA-2034 In modern cassandra, anything but a successful ack from a coordinated write results in a hint on the coordinator. > To solve your issue, as I mentioned, either do nodetool repair, or increase > your consistency level. By the way, you probably write faster than your > cluster can handle if you see that many dropped mutations. If his hints are ultimately delivered, OP should not "need" repair to be consistent. =Rob
So, if you want to grab by the created_at and occasionally limit by question id, that is why you'd use created_at. The way the primary keys work is the first part of the primary key is the Partioner key, that field is what essentially is the single cassandra row. The second key is the order preserving key, so you can sort by that key. If you have a third piece, then that is the secondary order preserving key. The reason you'd want to do (user_id, created_at, question_id) is because when you do a query on the keys, if you MUST use the preceding pieces of the primary key. So in your case, you could not do a query with just user_id and question_id with the user-created-question key. Alternatively if you went with (user_id, question_id, created_at), you would not be able to include a range of created_at unless you were also filtering on the question_id. Does that make sense? As for the large rows, 10k is unlikely to cause you too many issues (unless the answer is potentially a big blob of text). Newer versions of cassandra deal with a lot of things in far, far, superior ways to < 1.0. For a really good primary on keys in cql and how to potentially avoid hot rows, a really good article to read is this one: http://thelastpickle.com/2013/01/11/primary-keys-in-cql/ Aaron did a great job of laying out the subtleties of primary keys in CQL. On Wed, Jun 19, 2013 at 2:21 PM, Christopher J. Bottaro < cjbottaro@academicworks.com> wrote: > Interesting, thank you for the reply. > > Two questions though... > > Why should created_at come before question_id in the primary key? In > other words, why (user_id, created_at, question_id) instead of (user_id, > question_id, created_at)? > > Given this setup, all a user's answers (all 10k) will be stored in a > single C* (internal, not cql) row? I thought having "fat" or "big" rows > was bad. I worked with Cassandra 0.6 at my previous job and given the > nature of our work, we would sometimes generate these "fat" rows... at > which point Cassandra would basically shit the bed. > > Thanks for the help. > > > On Wed, Jun 19, 2013 at 12:26 PM, David McNelis <dmcnelis@gmail.com>wrote: > >> I think you'd just be better served with just a little different primary >> key. >> >> If your primary key was (user_id, created_at) or (user_id, created_at, >> question_id), then you'd be able to run the above query without a problem. >> >> This will mean that the entire pantheon of a specific user_id will be >> stored as a 'row' (in the old style C* vernacular), and then the >> information would be ordered by the 2nd piece of the primary key (or 2nd, >> then 3rd if you included question_id). >> >> You would certainly want to include any field that makes a record unique >> in the primary key. Another thing to note is that if a field is part of >> the primary key you can not create a secondary index on that field. You >> can work around that by storing the field twice, but you might want to >> rethink your structure if you find yourself doing that often. >> >> >> On Wed, Jun 19, 2013 at 12:05 PM, Christopher J. Bottaro < >> cjbottaro@academicworks.com> wrote: >> >>> Hello, >>> >>> We are considering using Cassandra and I want to make sure our use case >>> fits Cassandra's strengths. We have the table like: >>> >>> answers >>> ------- >>> user_id | question_id | result | created_at >>> >>> Where our most common query will be something like: >>> >>> SELECT * FROM answers WHERE user_id = 123 AND created_at > '01/01/2012' >>> AND created_at < '01/01/2013' >>> >>> Sometimes we will also limit by a question_id or a list of question_ids. >>> >>> Secondary indexes will be created on user_id and question_id. We expect >>> the upper bound of number of answers for a given user to be around 10,000. >>> >>> Now my understanding of how Cassandra will run the aforementioned query >>> is that it will load all the answers for a given user into memory using the >>> secondary index, then scan over that set filtering based on the dates. >>> >>> Considering that that will be our most used query and it will happen >>> very often, is this a bad use case for Cassandra? >>> >>> Thanks for the help. >>> >> >> >
Interesting, thank you for the reply. Two questions though... Why should created_at come before question_id in the primary key? In other words, why (user_id, created_at, question_id) instead of (user_id, question_id, created_at)? Given this setup, all a user's answers (all 10k) will be stored in a single C* (internal, not cql) row? I thought having "fat" or "big" rows was bad. I worked with Cassandra 0.6 at my previous job and given the nature of our work, we would sometimes generate these "fat" rows... at which point Cassandra would basically shit the bed. Thanks for the help. On Wed, Jun 19, 2013 at 12:26 PM, David McNelis <dmcnelis@gmail.com> wrote: > I think you'd just be better served with just a little different primary > key. > > If your primary key was (user_id, created_at) or (user_id, created_at, > question_id), then you'd be able to run the above query without a problem. > > This will mean that the entire pantheon of a specific user_id will be > stored as a 'row' (in the old style C* vernacular), and then the > information would be ordered by the 2nd piece of the primary key (or 2nd, > then 3rd if you included question_id). > > You would certainly want to include any field that makes a record unique > in the primary key. Another thing to note is that if a field is part of > the primary key you can not create a secondary index on that field. You > can work around that by storing the field twice, but you might want to > rethink your structure if you find yourself doing that often. > > > On Wed, Jun 19, 2013 at 12:05 PM, Christopher J. Bottaro < > cjbottaro@academicworks.com> wrote: > >> Hello, >> >> We are considering using Cassandra and I want to make sure our use case >> fits Cassandra's strengths. We have the table like: >> >> answers >> ------- >> user_id | question_id | result | created_at >> >> Where our most common query will be something like: >> >> SELECT * FROM answers WHERE user_id = 123 AND created_at > '01/01/2012' >> AND created_at < '01/01/2013' >> >> Sometimes we will also limit by a question_id or a list of question_ids. >> >> Secondary indexes will be created on user_id and question_id. We expect >> the upper bound of number of answers for a given user to be around 10,000. >> >> Now my understanding of how Cassandra will run the aforementioned query >> is that it will load all the answers for a given user into memory using the >> secondary index, then scan over that set filtering based on the dates. >> >> Considering that that will be our most used query and it will happen very >> often, is this a bad use case for Cassandra? >> >> Thanks for the help. >> > >
Hi,
I believe what he's recommending is:
CREATE TABLE count3 (
counter text,
ts timeuuid,
key1 text,
value int,
PRIMARY KEY (counter, ts)
)
That way counter will be your partitioning key, and all the rows that have the same counter
value will be clustered (stored as a single wide row sorted by the ts value). In this scenario
the query:
where counter = 'test' and ts > minTimeuuid('2013-06-18 16:23:00') and ts < minTimeuuid('2013-06-18
16:24:00');
would actually be a sequential read on a wide row on a single node.
--
Francisco Andrades Grassi
www.bigjocker.com
@bigjocker
On Jun 19, 2013, at 12:17 PM, "Ryan, Brent" <BRyan@cvent.com> wrote:
> Tyler,
>
> You're recommending this schema instead, correct?
>
> CREATE TABLE count3 (
> counter text,
> ts timeuuid,
> key1 text,
> value int,
> PRIMARY KEY (ts, counter)
> )
>
> I believe I tried this as well and ran into similar problems but I'll try it again.
I'm using the "ByteOrderedPartitioner" if that helps with the latest version of DSE community
edition which I believe is Cassandra 1.2.3.
>
>
> Thanks,
> Brent
>
>
> From: Tyler Hobbs <tyler@datastax.com>
> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Date: Wednesday, June 19, 2013 11:00 AM
> To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Subject: Re: timeuuid and cql3 query
>
>
> On Wed, Jun 19, 2013 at 8:08 AM, Ryan, Brent <BRyan@cvent.com> wrote:
>
> CREATE TABLE count3 (
> counter text,
> ts timeuuid,
> key1 text,
> value int,
> PRIMARY KEY ((counter, ts))
> )
>
> Instead of doing a composite partition key, remove a set of parens and let ts be your
clustering key. That will cause cql rows to be stored in sorted order by the ts column (for
a given value of "counter") and allow you to do the kind of query you're looking for.
>
>
> --
> Tyler Hobbs
> DataStax
You have a lot of Dropped Mutations which means those writes might not go through. Since you have CL.ONE as write consistency, your client doesn't see the exception if write fails only on one node. I think hints are only stored when the other node is down, not on the dropped mutations. (Correct me if I am wrong, actually it's not a bad idea to store hints for dropped mutations and replay them later?) To solve your issue, as I mentioned, either do nodetool repair, or increase your consistency level. By the way, you probably write faster than your cluster can handle if you see that many dropped mutations. -Wei ----- Original Message ----- From: "James Lee" <James.Lee@metaswitch.com> To: user@cassandra.apache.org Sent: Wednesday, June 19, 2013 2:22:39 AM Subject: RE: Data not fully replicated with 2 nodes and replication factor 2 The test tool I am using catches any exceptions on the original writes and resubmits the write request until it's successful (bailing out after 5 failures). So for each key Cassandra has reported a successful write. Nodetool says the following - I'm guessing the pending hinted handoff is the interesting bit? comet-mvs01:/dsc-cassandra-1.2.2# ./bin/nodetool tpstats Pool Name Active Pending Completed Blocked All time blocked ReadStage 0 0 35445 0 0 RequestResponseStage 0 0 1535171 0 0 MutationStage 0 0 3038941 0 0 ReadRepairStage 0 0 2695 0 0 ReplicateOnWriteStage 0 0 0 0 0 GossipStage 0 0 2898 0 0 AntiEntropyStage 0 0 0 0 0 MigrationStage 0 0 245 0 0 MemtablePostFlusher 0 0 1260 0 0 FlushWriter 0 0 633 0 212 MiscStage 0 0 0 0 0 commitlog_archiver 0 0 0 0 0 InternalResponseStage 0 0 0 0 0 HintedHandoff 1 1 0 0 0 Message type Dropped RANGE_SLICE 0 READ_REPAIR 0 BINARY 0 READ 0 MUTATION 60427 _TRACE 0 REQUEST_RESPONSE 0 Looking at the hints column family in the system keyspace, I see one row with a large number of columns. Presumably that along with the nodetool output above suggests there are hinted handoffs pending? How long should I expect these to remain for? Ah, actually now that I re-run the command it seems that nodetool now reports that hint as completed and there are no hints left in the system keyspace on either node. I'm still seeing failures to read the data I'm expecting though, as before. Note that I've run this with a smaller data set (2M rows, 1GB data total) for this latest test. Thanks, James -----Original Message----- From: Robert Coli [mailto:rcoli@eventbrite.com] Sent: 18 June 2013 19:45 To: user@cassandra.apache.org Subject: Re: Data not fully replicated with 2 nodes and replication factor 2 On Tue, Jun 18, 2013 at 11:36 AM, Wei Zhu <wz1975@yahoo.com> wrote: > Cassandra doesn't do async replication like HBase does.You can run > nodetool repair to insure the consistency. While this answer is true, it is somewhat non-responsive to the OP. If the OP didn't see timeout exception, the theoretical worst case is that he should have hints stored for initially failed to replicate writes. His nodes should not be failing GC with a total data size of 5gb on an 8gb heap, so those hints should deliver quite quickly. After 30 minutes those hints should certainly be delivered. @OP : do you see hints being stored? does nodetool tpstats indicate dropped messages? =Rob
>
> On its face my answer is "not... really"? What do you view yourself as
> getting with this technique versus using built in replication? As an
> example, you lose the ability to do LOCAL_QUORUM vs EACH_QUORUM
> consistency level operations?
Doing replication manually sounds like a recipe for the DC's eventually
getting subtly out of sync with each other. If a connection goes down
between DC's, and you are taking data at both, how will you catch each
other up? C* already offers that resolution for you, and you'd have to
work pretty hard to reproduce it for no obvious benefit that I can see.
For minimum effort, definitely rely on Cassandra's well-tested codebase for
this.
On Wed, Jun 19, 2013 at 2:27 PM, Robert Coli <rcoli@eventbrite.com> wrote:
> On Wed, Jun 19, 2013 at 10:50 AM, Faraaz Sareshwala
> <fsareshwala@quantcast.com> wrote:
> > Each datacenter will have a cassandra cluster with a separate set of
> seeds
> > specific to that datacenter. However, the cluster name will be the same.
> >
> > Question 1: is this enough to guarentee that the three datacenters will
> have
> > distinct cassandra clusters as well? Or will one node in datacenter A
> still
> > somehow be able to join datacenter B's ring.
>
> If they have network connectivity and the same cluster name, they are
> the same logical cluster. However if your nodes share tokens and you
> have auto_bootstrap=yes (the implicit default) the second node you
> attempt to start will refuse to start because you are trying to
> bootstrap it into the range of a live node.
>
> > For now, we are planning on using our own relay mechanism to transfer
> > data changes from one datacenter to another.
>
> Are you planning to use the streaming commitlog functionality for
> this? Not sure how you would capture all changes otherwise, except
> having your app just write the same thing to multiple places? Unless
> data timestamps are identical between clusters, otherwise identical
> data will not merge properly, as cassandra uses data timestamps to
> merge.
>
> > Question 2: is this a sane strategy?
>
> On its face my answer is "not... really"? What do you view yourself as
> getting with this technique versus using built in replication? As an
> example, you lose the ability to do LOCAL_QUORUM vs EACH_QUORUM
> consistency level operations?
>
> > Question 3: eventually, we want to turn all these cassandra clusters
> into one
> > large multi-datacenter cluster. What's the best practice to do this?
> Should I
> > just add nodes from all datacenters to the list of seeds and let
> cassandra
> > resolve differences? Is there another way I don't know about?
>
> If you are using NetworkTopologyStrategy and have the same cluster
> name for your isolated clusters, all you need to do is :
>
> 1) configure NTS to store replicas on a per-datacenter basis
> 2) ensure that your nodes are in different logical data centers (by
> default, all nodes are in DC1/rack1)
> 3) ensure that clusters are able to reach each other
> 4) ensure that tokens do not overlap between clusters (the common
> technique with manual token assignment is that each node gets a range
> which is off-by-one)
> 5) ensure that all nodes seed lists contain (recommended) 3 seeds from
> each DC
> 6) rolling restart (so the new seed list is picked up)
> 7) repair ("should" only be required if writes have not replicated via
> your out of band mechanism)
>
> Vnodes change the picture slightly because the chance of your clusters
> having conflicting tokens increases with the number of token ranges
> you have.
>
> =Rob
>
If you want, you can try to force the GC through Jconsole. Memory->Perform GC. It theoretically triggers a full GC and when it will happen depends on the JVM -Wei ----- Original Message ----- From: "Robert Coli" <rcoli@eventbrite.com> To: user@cassandra.apache.org Sent: Tuesday, June 18, 2013 10:43:13 AM Subject: Re: Heap is not released and streaming hangs at 0% On Tue, Jun 18, 2013 at 10:33 AM, srmore <comomore@gmail.com> wrote: > But then shouldn't JVM C G it eventually ? I can still see Cassandra alive > and kicking but looks like the heap is locked up even after the traffic is > long stopped. No, when GC system fails this hard it is often a permanent failure which requires a restart of the JVM. > nodetool -h localhost flush didn't do much good. This adds support to the idea that your heap is too full, and not full of memtables. You could try nodetool -h localhost invalidatekeycache, but that probably will not free enough memory to help you. =Rob
On Wed, Jun 19, 2013 at 10:50 AM, Faraaz Sareshwala
<fsareshwala@quantcast.com> wrote:
> Each datacenter will have a cassandra cluster with a separate set of seeds
> specific to that datacenter. However, the cluster name will be the same.
>
> Question 1: is this enough to guarentee that the three datacenters will have
> distinct cassandra clusters as well? Or will one node in datacenter A still
> somehow be able to join datacenter B's ring.
If they have network connectivity and the same cluster name, they are
the same logical cluster. However if your nodes share tokens and you
have auto_bootstrap=yes (the implicit default) the second node you
attempt to start will refuse to start because you are trying to
bootstrap it into the range of a live node.
> For now, we are planning on using our own relay mechanism to transfer
> data changes from one datacenter to another.
Are you planning to use the streaming commitlog functionality for
this? Not sure how you would capture all changes otherwise, except
having your app just write the same thing to multiple places? Unless
data timestamps are identical between clusters, otherwise identical
data will not merge properly, as cassandra uses data timestamps to
merge.
> Question 2: is this a sane strategy?
On its face my answer is "not... really"? What do you view yourself as
getting with this technique versus using built in replication? As an
example, you lose the ability to do LOCAL_QUORUM vs EACH_QUORUM
consistency level operations?
> Question 3: eventually, we want to turn all these cassandra clusters into one
> large multi-datacenter cluster. What's the best practice to do this? Should I
> just add nodes from all datacenters to the list of seeds and let cassandra
> resolve differences? Is there another way I don't know about?
If you are using NetworkTopologyStrategy and have the same cluster
name for your isolated clusters, all you need to do is :
1) configure NTS to store replicas on a per-datacenter basis
2) ensure that your nodes are in different logical data centers (by
default, all nodes are in DC1/rack1)
3) ensure that clusters are able to reach each other
4) ensure that tokens do not overlap between clusters (the common
technique with manual token assignment is that each node gets a range
which is off-by-one)
5) ensure that all nodes seed lists contain (recommended) 3 seeds from each DC
6) rolling restart (so the new seed list is picked up)
7) repair ("should" only be required if writes have not replicated via
your out of band mechanism)
Vnodes change the picture slightly because the chance of your clusters
having conflicting tokens increases with the number of token ranges
you have.
=Rob
So part of it is a bug, namely
https://issues.apache.org/jira/browse/CASSANDRA-5666. In summary CQL3
should not accept: ts > minTimeuuid('2013-06-17 22:36:16') and ts <
minTimeuuid('2013-06-20 22:44:02'), because it does no know how to handle
it properly. What it should support is token(ts) >
token(minTimeuuid('2013-06-17 22:36:16')) and token(ts) <
token(minTimeuuid('2013-06-20 22:44:02')). And that is different because
the token always sort by bytes, and comparing timeuuid by bytes does not
yield a time based ordering.
Long story short, using non-equal condition on the partition key (i.e. the
first part of your primary key) is generally not advised. Or to put it
another way, the use of the byte ordering partitioner is discouraged. But
if you still want to use the ordering partitioner and do range queries on
the partition key, do not use a timeuuid, because the ordering that the
partitioner enforce will not be one that is meaningful (due to the timeuuid
layout).
--
Sylvain
On Wed, Jun 19, 2013 at 7:04 PM, Ryan, Brent <BRyan@cvent.com> wrote:
> Note that it seems to work when you structure your schema in this
> example below, BUT this is a problem because all of my data will wind up
> hitting a single node in my cassandra cluster because the partitioning key
> is "counter" and that isn't unique enough. I was hoping that I wasn't
> going to need to build up my own "sharding" scheme as this blog talks about
> (http://www.datastax.com/dev/blog/advanced-time-series-with-cassandra)
> because this becomes much harder for other clients to integrate with
> because they now need to know how my data is structured in order to get it
> out.
>
> CREATE TABLE count5 (
> counter text,
> ts timeuuid,
> key1 text,
> value int,
> PRIMARY KEY (counter, ts)
> ) WITH
> bloom_filter_fp_chance=0.010000 AND
> caching='KEYS_ONLY' AND
> comment='' AND
> dclocal_read_repair_chance=0.000000 AND
> gc_grace_seconds=864000 AND
> read_repair_chance=0.100000 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'SnappyCompressor'};
>
> cqlsh:Test> select counter,dateof(ts),key1,value from count5 where
> counter = 'test' and ts > minTimeuuid('2013-06-17 22:36:16') and ts <
> minTimeuuid('2013-06-18 22:44:02');
>
> counter | dateof(ts) | key1 | value
> ---------+--------------------------+------+-------
> test | 2013-06-18 22:43:53-0400 | 1 | 1
> test | 2013-06-18 22:43:54-0400 | 1 | 1
> test | 2013-06-18 22:43:55-0400 | 1 | 1
> test | 2013-06-18 22:43:56-0400 | 1 | 1
> test | 2013-06-18 22:43:58-0400 | 1 | 1
> test | 2013-06-18 22:43:58-0400 | 1 | 1
> test | 2013-06-18 22:43:59-0400 | 1 | 1
> test | 2013-06-18 22:44:00-0400 | 1 | 1
> test | 2013-06-18 22:44:01-0400 | 1 | 1
>
> cqlsh:Test> select counter,dateof(ts),key1,value from count5 where
> counter = 'test' and ts > minTimeuuid('2013-06-17 22:36:16') and ts <
> minTimeuuid('2013-06-20 22:44:02');
>
> counter | dateof(ts) | key1 | value
> ---------+--------------------------+------+-------
> test | 2013-06-18 22:43:53-0400 | 1 | 1
> test | 2013-06-18 22:43:54-0400 | 1 | 1
> test | 2013-06-18 22:43:55-0400 | 1 | 1
> test | 2013-06-18 22:43:56-0400 | 1 | 1
> test | 2013-06-18 22:43:58-0400 | 1 | 1
> test | 2013-06-18 22:43:58-0400 | 1 | 1
> test | 2013-06-18 22:43:59-0400 | 1 | 1
> test | 2013-06-18 22:44:00-0400 | 1 | 1
> test | 2013-06-18 22:44:01-0400 | 1 | 1
> test | 2013-06-18 22:44:02-0400 | 1 | 1
> test | 2013-06-18 22:44:02-0400 | 1 | 1
> test | 2013-06-18 22:44:03-0400 | 1 | 1
> test | 2013-06-18 22:44:04-0400 | 1 | 1
> test | 2013-06-18 22:44:05-0400 | 1 | 1
> test | 2013-06-18 22:44:06-0400 | 1 | 1
>
>
> From: <Ryan>, Brent Ryan <bryan@cvent.com>
> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Date: Wednesday, June 19, 2013 12:56 PM
>
> To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Subject: Re: timeuuid and cql3 query
>
> Here's an example of that not working:
>
> cqlsh:Test> desc table count4;
>
> CREATE TABLE count4 (
> ts timeuuid,
> counter text,
> key1 text,
> value int,
> PRIMARY KEY (ts, counter)
> ) WITH
> bloom_filter_fp_chance=0.010000 AND
> caching='KEYS_ONLY' AND
> comment='' AND
> dclocal_read_repair_chance=0.000000 AND
> gc_grace_seconds=864000 AND
> read_repair_chance=0.100000 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'SnappyCompressor'};
>
> cqlsh:Test> select counter,dateof(ts),key1,value from count4;
>
> counter | dateof(ts) | key1 | value
> ---------+--------------------------+------+-------
> test | 2013-06-18 22:36:16-0400 | 1 | 1
> test | 2013-06-18 22:36:18-0400 | 1 | 1
> test | 2013-06-18 22:36:18-0400 | 1 | 1
> test | 2013-06-18 22:36:18-0400 | 1 | 1
> test | 2013-06-18 22:36:19-0400 | 1 | 1
> test | 2013-06-18 22:36:19-0400 | 1 | 1
> test | 2013-06-18 22:36:20-0400 | 1 | 1
> test | 2013-06-18 22:36:20-0400 | 1 | 1
> test | 2013-06-18 22:36:21-0400 | 1 | 1
> test | 2013-06-18 22:36:21-0400 | 1 | 1
> test | 2013-06-18 22:36:22-0400 | 1 | 1
> test | 2013-06-18 22:36:22-0400 | 1 | 1
> test | 2013-06-18 22:36:23-0400 | 1 | 1
> test | 2013-06-18 22:36:23-0400 | 1 | 1
> test | 2013-06-18 22:36:25-0400 | 1 | 1
> test | 2013-06-18 22:36:27-0400 | 1 | 1
> test | 2013-06-18 22:36:28-0400 | 1 | 1
>
> cqlsh:Statistics> select counter,dateof(ts),key1,value from count4 where
> ts > minTimeuuid('2013-06-17 22:36:16') and ts < minTimeuuid('2013-06-19
> 22:36:20');
> Bad Request: 2 Start key must sort before (or equal to) finish key in your
> partitioner!
>
>
>
> Any ideas? Seems like a bug to me, right?
>
> Brent
>
> From: <Ryan>, Brent Ryan <bryan@cvent.com>
> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Date: Wednesday, June 19, 2013 12:47 PM
> To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Subject: Re: timeuuid and cql3 query
>
> Tyler,
>
> You're recommending this schema instead, correct?
>
> CREATE TABLE count3 (
> counter text,
> ts timeuuid,
> key1 text,
> value int,
> PRIMARY KEY (ts, counter)
> )
>
> I believe I tried this as well and ran into similar problems but I'll
> try it again. I'm using the "ByteOrderedPartitioner" if that helps with
> the latest version of DSE community edition which I believe is Cassandra
> 1.2.3.
>
>
> Thanks,
> Brent
>
>
> From: Tyler Hobbs <tyler@datastax.com>
> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Date: Wednesday, June 19, 2013 11:00 AM
> To: "user@cassandra.apache.org" <user@cassandra.apache.org>
> Subject: Re: timeuuid and cql3 query
>
>
> On Wed, Jun 19, 2013 at 8:08 AM, Ryan, Brent <BRyan@cvent.com> wrote:
>
>>
>> CREATE TABLE count3 (
>> counter text,
>> ts timeuuid,
>> key1 text,
>> value int,
>> PRIMARY KEY ((counter, ts))
>> )
>>
>
> Instead of doing a composite partition key, remove a set of parens and let
> ts be your clustering key. That will cause cql rows to be stored in sorted
> order by the ts column (for a given value of "counter") and allow you to do
> the kind of query you're looking for.
>
>
> --
> Tyler Hobbs
> DataStax <http://datastax.com/>
>
My company is planning on deploying cassandra to three separate datacenters. Each datacenter will have a cassandra cluster with a separate set of seeds specific to that datacenter. However, the cluster name will be the same. Question 1: is this enough to guarentee that the three datacenters will have distinct cassandra clusters as well? Or will one node in datacenter A still somehow be able to join datacenter B's ring. Cassandra has cross datacenter replication and we plan to use that in the future. For now, we are planning on using our own relay mechanism to transfer data changes from one datacenter to another. Each cassandra cluster in each datacenter will have the same keyspaces and column families with the same schema. Datacenter A will send mutations over this relay to datacenter B which will replay the mutation in cassandra. Therefore, datacenter A's cassandra cluster will look identical to datacenter B's cassandra cluster, but not through the cross datacenter replication that cassandra offers. Question 2: is this a sane strategy? We're trying to make the smallest possible change when deploying cassandra. Our plan is to slowly move our infrastructure over to relying more on cassandra once we can assess how it behaves with our workload. Question 3: eventually, we want to turn all these cassandra clusters into one large multi-datacenter cluster. What's the best practice to do this? Should I just add nodes from all datacenters to the list of seeds and let cassandra resolve differences? Is there another way I don't know about? Thank you, Faraaz
On Wed, Jun 19, 2013 at 5:47 AM, Michal Michalski <michalm@opera.com> wrote: > You can also perform a major compaction via nodetool compact (for > SizeTieredCompaction), but - again - you really should not do it unless > you're really sure what you do, as it compacts all the SSTables together, > which is not something you might want to achieve in most of the cases. If you do that and discover you did not want to : https://github.com/pcmanus/cassandra/tree/sstable_split Will enable you to split your monolithic sstable back into smaller sstables. =Rob PS - @pcmanus, here's that reminder we discussed @ summit to merge this tool into upstream! :D
I think you'd just be better served with just a little different primary key. If your primary key was (user_id, created_at) or (user_id, created_at, question_id), then you'd be able to run the above query without a problem. This will mean that the entire pantheon of a specific user_id will be stored as a 'row' (in the old style C* vernacular), and then the information would be ordered by the 2nd piece of the primary key (or 2nd, then 3rd if you included question_id). You would certainly want to include any field that makes a record unique in the primary key. Another thing to note is that if a field is part of the primary key you can not create a secondary index on that field. You can work around that by storing the field twice, but you might want to rethink your structure if you find yourself doing that often. On Wed, Jun 19, 2013 at 12:05 PM, Christopher J. Bottaro < cjbottaro@academicworks.com> wrote: > Hello, > > We are considering using Cassandra and I want to make sure our use case > fits Cassandra's strengths. We have the table like: > > answers > ------- > user_id | question_id | result | created_at > > Where our most common query will be something like: > > SELECT * FROM answers WHERE user_id = 123 AND created_at > '01/01/2012' > AND created_at < '01/01/2013' > > Sometimes we will also limit by a question_id or a list of question_ids. > > Secondary indexes will be created on user_id and question_id. We expect > the upper bound of number of answers for a given user to be around 10,000. > > Now my understanding of how Cassandra will run the aforementioned query is > that it will load all the answers for a given user into memory using the > secondary index, then scan over that set filtering based on the dates. > > Considering that that will be our most used query and it will happen very > often, is this a bad use case for Cassandra? > > Thanks for the help. >
Hello, We are considering using Cassandra and I want to make sure our use case fits Cassandra's strengths. We have the table like: answers ------- user_id | question_id | result | created_at Where our most common query will be something like: SELECT * FROM answers WHERE user_id = 123 AND created_at > '01/01/2012' AND created_at < '01/01/2013' Sometimes we will also limit by a question_id or a list of question_ids. Secondary indexes will be created on user_id and question_id. We expect the upper bound of number of answers for a given user to be around 10,000. Now my understanding of how Cassandra will run the aforementioned query is that it will load all the answers for a given user into memory using the secondary index, then scan over that set filtering based on the dates. Considering that that will be our most used query and it will happen very often, is this a bad use case for Cassandra? Thanks for the help.
Note that it seems to work when you structure your schema in this example below, BUT this is
a problem because all of my data will wind up hitting a single node in my cassandra cluster
because the partitioning key is "counter" and that isn't unique enough. I was hoping that
I wasn't going to need to build up my own "sharding" scheme as this blog talks about (http://www.datastax.com/dev/blog/advanced-time-series-with-cassandra)
because this becomes much harder for other clients to integrate with because they now need
to know how my data is structured in order to get it out.
CREATE TABLE count5 (
counter text,
ts timeuuid,
key1 text,
value int,
PRIMARY KEY (counter, ts)
) WITH
bloom_filter_fp_chance=0.010000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.100000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};
cqlsh:Test> select counter,dateof(ts),key1,value from count5 where counter = 'test' and
ts > minTimeuuid('2013-06-17 22:36:16') and ts < minTimeuuid('2013-06-18 22:44:02');
counter | dateof(ts) | key1 | value
---------+--------------------------+------+-------
test | 2013-06-18 22:43:53-0400 | 1 | 1
test | 2013-06-18 22:43:54-0400 | 1 | 1
test | 2013-06-18 22:43:55-0400 | 1 | 1
test | 2013-06-18 22:43:56-0400 | 1 | 1
test | 2013-06-18 22:43:58-0400 | 1 | 1
test | 2013-06-18 22:43:58-0400 | 1 | 1
test | 2013-06-18 22:43:59-0400 | 1 | 1
test | 2013-06-18 22:44:00-0400 | 1 | 1
test | 2013-06-18 22:44:01-0400 | 1 | 1
cqlsh:Test> select counter,dateof(ts),key1,value from count5 where counter = 'test' and
ts > minTimeuuid('2013-06-17 22:36:16') and ts < minTimeuuid('2013-06-20 22:44:02');
counter | dateof(ts) | key1 | value
---------+--------------------------+------+-------
test | 2013-06-18 22:43:53-0400 | 1 | 1
test | 2013-06-18 22:43:54-0400 | 1 | 1
test | 2013-06-18 22:43:55-0400 | 1 | 1
test | 2013-06-18 22:43:56-0400 | 1 | 1
test | 2013-06-18 22:43:58-0400 | 1 | 1
test | 2013-06-18 22:43:58-0400 | 1 | 1
test | 2013-06-18 22:43:59-0400 | 1 | 1
test | 2013-06-18 22:44:00-0400 | 1 | 1
test | 2013-06-18 22:44:01-0400 | 1 | 1
test | 2013-06-18 22:44:02-0400 | 1 | 1
test | 2013-06-18 22:44:02-0400 | 1 | 1
test | 2013-06-18 22:44:03-0400 | 1 | 1
test | 2013-06-18 22:44:04-0400 | 1 | 1
test | 2013-06-18 22:44:05-0400 | 1 | 1
test | 2013-06-18 22:44:06-0400 | 1 | 1
From: <Ryan>, Brent Ryan <bryan@cvent.com<mailto:bryan@cvent.com>>
Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Date: Wednesday, June 19, 2013 12:56 PM
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: Re: timeuuid and cql3 query
Here's an example of that not working:
cqlsh:Test> desc table count4;
CREATE TABLE count4 (
ts timeuuid,
counter text,
key1 text,
value int,
PRIMARY KEY (ts, counter)
) WITH
bloom_filter_fp_chance=0.010000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.100000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};
cqlsh:Test> select counter,dateof(ts),key1,value from count4;
counter | dateof(ts) | key1 | value
---------+--------------------------+------+-------
test | 2013-06-18 22:36:16-0400 | 1 | 1
test | 2013-06-18 22:36:18-0400 | 1 | 1
test | 2013-06-18 22:36:18-0400 | 1 | 1
test | 2013-06-18 22:36:18-0400 | 1 | 1
test | 2013-06-18 22:36:19-0400 | 1 | 1
test | 2013-06-18 22:36:19-0400 | 1 | 1
test | 2013-06-18 22:36:20-0400 | 1 | 1
test | 2013-06-18 22:36:20-0400 | 1 | 1
test | 2013-06-18 22:36:21-0400 | 1 | 1
test | 2013-06-18 22:36:21-0400 | 1 | 1
test | 2013-06-18 22:36:22-0400 | 1 | 1
test | 2013-06-18 22:36:22-0400 | 1 | 1
test | 2013-06-18 22:36:23-0400 | 1 | 1
test | 2013-06-18 22:36:23-0400 | 1 | 1
test | 2013-06-18 22:36:25-0400 | 1 | 1
test | 2013-06-18 22:36:27-0400 | 1 | 1
test | 2013-06-18 22:36:28-0400 | 1 | 1
cqlsh:Statistics> select counter,dateof(ts),key1,value from count4 where ts > minTimeuuid('2013-06-17
22:36:16') and ts < minTimeuuid('2013-06-19 22:36:20');
Bad Request: 2 Start key must sort before (or equal to) finish key in your partitioner!
Any ideas? Seems like a bug to me, right?
Brent
From: <Ryan>, Brent Ryan <bryan@cvent.com<mailto:bryan@cvent.com>>
Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Date: Wednesday, June 19, 2013 12:47 PM
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: Re: timeuuid and cql3 query
Tyler,
You're recommending this schema instead, correct?
CREATE TABLE count3 (
counter text,
ts timeuuid,
key1 text,
value int,
PRIMARY KEY (ts, counter)
)
I believe I tried this as well and ran into similar problems but I'll try it again. I'm using
the "ByteOrderedPartitioner" if that helps with the latest version of DSE community edition
which I believe is Cassandra 1.2.3.
Thanks,
Brent
From: Tyler Hobbs <tyler@datastax.com<mailto:tyler@datastax.com>>
Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Date: Wednesday, June 19, 2013 11:00 AM
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: Re: timeuuid and cql3 query
On Wed, Jun 19, 2013 at 8:08 AM, Ryan, Brent <BRyan@cvent.com<mailto:BRyan@cvent.com>>
wrote:
CREATE TABLE count3 (
counter text,
ts timeuuid,
key1 text,
value int,
PRIMARY KEY ((counter, ts))
)
Instead of doing a composite partition key, remove a set of parens and let ts be your clustering
key. That will cause cql rows to be stored in sorted order by the ts column (for a given
value of "counter") and allow you to do the kind of query you're looking for.
--
Tyler Hobbs
DataStax<http://datastax.com/>
Here's an example of that not working:
cqlsh:Test> desc table count4;
CREATE TABLE count4 (
ts timeuuid,
counter text,
key1 text,
value int,
PRIMARY KEY (ts, counter)
) WITH
bloom_filter_fp_chance=0.010000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.100000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};
cqlsh:Test> select counter,dateof(ts),key1,value from count4;
counter | dateof(ts) | key1 | value
---------+--------------------------+------+-------
test | 2013-06-18 22:36:16-0400 | 1 | 1
test | 2013-06-18 22:36:18-0400 | 1 | 1
test | 2013-06-18 22:36:18-0400 | 1 | 1
test | 2013-06-18 22:36:18-0400 | 1 | 1
test | 2013-06-18 22:36:19-0400 | 1 | 1
test | 2013-06-18 22:36:19-0400 | 1 | 1
test | 2013-06-18 22:36:20-0400 | 1 | 1
test | 2013-06-18 22:36:20-0400 | 1 | 1
test | 2013-06-18 22:36:21-0400 | 1 | 1
test | 2013-06-18 22:36:21-0400 | 1 | 1
test | 2013-06-18 22:36:22-0400 | 1 | 1
test | 2013-06-18 22:36:22-0400 | 1 | 1
test | 2013-06-18 22:36:23-0400 | 1 | 1
test | 2013-06-18 22:36:23-0400 | 1 | 1
test | 2013-06-18 22:36:25-0400 | 1 | 1
test | 2013-06-18 22:36:27-0400 | 1 | 1
test | 2013-06-18 22:36:28-0400 | 1 | 1
cqlsh:Statistics> select counter,dateof(ts),key1,value from count4 where ts > minTimeuuid('2013-06-17
22:36:16') and ts < minTimeuuid('2013-06-19 22:36:20');
Bad Request: 2 Start key must sort before (or equal to) finish key in your partitioner!
Any ideas? Seems like a bug to me, right?
Brent
From: <Ryan>, Brent Ryan <bryan@cvent.com<mailto:bryan@cvent.com>>
Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Date: Wednesday, June 19, 2013 12:47 PM
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: Re: timeuuid and cql3 query
Tyler,
You're recommending this schema instead, correct?
CREATE TABLE count3 (
counter text,
ts timeuuid,
key1 text,
value int,
PRIMARY KEY (ts, counter)
)
I believe I tried this as well and ran into similar problems but I'll try it again. I'm using
the "ByteOrderedPartitioner" if that helps with the latest version of DSE community edition
which I believe is Cassandra 1.2.3.
Thanks,
Brent
From: Tyler Hobbs <tyler@datastax.com<mailto:tyler@datastax.com>>
Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Date: Wednesday, June 19, 2013 11:00 AM
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: Re: timeuuid and cql3 query
On Wed, Jun 19, 2013 at 8:08 AM, Ryan, Brent <BRyan@cvent.com<mailto:BRyan@cvent.com>>
wrote:
CREATE TABLE count3 (
counter text,
ts timeuuid,
key1 text,
value int,
PRIMARY KEY ((counter, ts))
)
Instead of doing a composite partition key, remove a set of parens and let ts be your clustering
key. That will cause cql rows to be stored in sorted order by the ts column (for a given
value of "counter") and allow you to do the kind of query you're looking for.
--
Tyler Hobbs
DataStax<http://datastax.com/>
Tyler, You're recommending this schema instead, correct? CREATE TABLE count3 ( counter text, ts timeuuid, key1 text, value int, PRIMARY KEY (ts, counter) ) I believe I tried this as well and ran into similar problems but I'll try it again. I'm using the "ByteOrderedPartitioner" if that helps with the latest version of DSE community edition which I believe is Cassandra 1.2.3. Thanks, Brent From: Tyler Hobbs <tyler@datastax.com<mailto:tyler@datastax.com>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Date: Wednesday, June 19, 2013 11:00 AM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> Subject: Re: timeuuid and cql3 query On Wed, Jun 19, 2013 at 8:08 AM, Ryan, Brent <BRyan@cvent.com<mailto:BRyan@cvent.com>> wrote: CREATE TABLE count3 ( counter text, ts timeuuid, key1 text, value int, PRIMARY KEY ((counter, ts)) ) Instead of doing a composite partition key, remove a set of parens and let ts be your clustering key. That will cause cql rows to be stored in sorted order by the ts column (for a given value of "counter") and allow you to do the kind of query you're looking for. -- Tyler Hobbs DataStax<http://datastax.com/>
How much data do you have per node? How much RAM per node? How much CPU per node? What is the avg CPU and memory usage? On Wed, Jun 19, 2013 at 12:16 AM, Joel Samuelsson <samuelsson.joel@gmail.com > wrote: > My Cassandra ps info: > > root 26791 1 0 07:14 ? 00:00:00 /usr/bin/jsvc -user > cassandra -home /opt/java/64/jre1.6.0_32/bin/../ -pidfile > /var/run/cassandra.pid -errfile &1 -outfile /var/log/cassandra/output.log > -cp > /usr/share/cassandra/lib/antlr-3.2.jar:/usr/share/cassandra/lib/avro-1.4.0-fixes.jar:/usr/share/cassandra/lib/avro-1.4.0-sources-fixes.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang-2.6.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.3.jar:/usr/share/cassandra/lib/guava-13.0.1.jar:/usr/share/cassandra/lib/high-scale-lib-1.1.2.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jline-1.0.jar:/usr/share/cassandra/lib/jna.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.7.0.jar:/usr/share/cassandra/lib/log4j-1.2.16.jar:/usr/share/cassandra/lib/lz4-1.1.0.jar:/usr/share/cassandra/lib/metrics-core-2.0.3.jar:/usr/share/cassandra/lib/netty-3.5.9.Final.jar:/usr/share/cassandra/lib/servlet-api-2.5-20081211.jar:/usr/share/cassandra/lib/slf4j-api-1.7.2.jar:/usr/share/cassandra/lib/slf4j-log4j12-1.7.2.jar:/usr/share/cassandra/lib/snakeyaml-1.6.jar:/usr/share/cassandra/lib/snappy-java-1.0.4.1.jar:/usr/share/cassandra/lib/snaptree-0.1.jar:/usr/share/cassandra/apache-cassandra-1.2.5.jar:/usr/share/cassandra/apache-cassandra-thrift-1.2.5.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar:/usr/share/java/jna.jar:/etc/cassandra:/usr/share/java/commons-daemon.jar > -Dlog4j.configuration=log4j-server.properties > -Dlog4j.defaultInitOverride=true > -XX:HeapDumpPath=/var/lib/cassandra/java_1371626058.hprof > -XX:ErrorFile=/var/lib/cassandra/hs_err_1371626058.log -ea > -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities > -XX:ThreadPriorityPolicy=42 -Xms4004M -Xmx4004M -Xmn800M > -XX:+HeapDumpOnOutOfMemoryError -Xss180k -XX:+UseParNewGC > -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 > -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 > -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseTLAB > -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.port=7199 > -Dcom.sun.management.jmxremote.ssl=false > -Dcom.sun.management.jmxremote.authenticate=false > org.apache.cassandra.service.CassandraDaemon > 103 26792 26791 99 07:14 ? 854015-22:02:22 /usr/bin/jsvc -user > cassandra -home /opt/java/64/jre1.6.0_32/bin/../ -pidfile > /var/run/cassandra.pid -errfile &1 -outfile /var/log/cassandra/output.log > -cp > /usr/share/cassandra/lib/antlr-3.2.jar:/usr/share/cassandra/lib/avro-1.4.0-fixes.jar:/usr/share/cassandra/lib/avro-1.4.0-sources-fixes.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang-2.6.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.3.jar:/usr/share/cassandra/lib/guava-13.0.1.jar:/usr/share/cassandra/lib/high-scale-lib-1.1.2.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.2.5.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jline-1.0.jar:/usr/share/cassandra/lib/jna.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.7.0.jar:/usr/share/cassandra/lib/log4j-1.2.16.jar:/usr/share/cassandra/lib/lz4-1.1.0.jar:/usr/share/cassandra/lib/metrics-core-2.0.3.jar:/usr/share/cassandra/lib/netty-3.5.9.Final.jar:/usr/share/cassandra/lib/servlet-api-2.5-20081211.jar:/usr/share/cassandra/lib/slf4j-api-1.7.2.jar:/usr/share/cassandra/lib/slf4j-log4j12-1.7.2.jar:/usr/share/cassandra/lib/snakeyaml-1.6.jar:/usr/share/cassandra/lib/snappy-java-1.0.4.1.jar:/usr/share/cassandra/lib/snaptree-0.1.jar:/usr/share/cassandra/apache-cassandra-1.2.5.jar:/usr/share/cassandra/apache-cassandra-thrift-1.2.5.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar:/usr/share/java/jna.jar:/etc/cassandra:/usr/share/java/commons-daemon.jar > -Dlog4j.configuration=log4j-server.properties > -Dlog4j.defaultInitOverride=true > -XX:HeapDumpPath=/var/lib/cassandra/java_1371626058.hprof > -XX:ErrorFile=/var/lib/cassandra/hs_err_1371626058.log -ea > -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar -XX:+UseThreadPriorities > -XX:ThreadPriorityPolicy=42 -Xms4004M -Xmx4004M -Xmn800M > -XX:+HeapDumpOnOutOfMemoryError -Xss180k -XX:+UseParNewGC > -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 > -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 > -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseTLAB > -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote.port=7199 > -Dcom.sun.management.jmxremote.ssl=false > -Dcom.sun.management.jmxremote.authenticate=false > org.apache.cassandra.service.CassandraDaemon > > Is it normal to have two processes like this? > >
I'm using the byte ordered partitioner. Sent from my iPhone On Jun 19, 2013, at 11:26 AM, "Sylvain Lebresne" <sylvain@datastax.com<mailto:sylvain@datastax.com>> wrote: You're using the ordered partitioner, right? On Wed, Jun 19, 2013 at 5:06 PM, Davide Anastasia <davide.anastasia@gmail.com<mailto:davide.anastasia@gmail.com>> wrote: Hi Tyler, I am interested in this scenario as well: could you please elaborate further your answer? Thanks a lot, Davide On 19 Jun 2013 16:01, "Tyler Hobbs" <tyler@datastax.com<mailto:tyler@datastax.com>> wrote: On Wed, Jun 19, 2013 at 8:08 AM, Ryan, Brent <BRyan@cvent.com<mailto:BRyan@cvent.com>> wrote: CREATE TABLE count3 ( counter text, ts timeuuid, key1 text, value int, PRIMARY KEY ((counter, ts)) ) Instead of doing a composite partition key, remove a set of parens and let ts be your clustering key. That will cause cql rows to be stored in sorted order by the ts column (for a given value of "counter") and allow you to do the kind of query you're looking for. -- Tyler Hobbs DataStax<http://datastax.com/>
You're using the ordered partitioner, right? On Wed, Jun 19, 2013 at 5:06 PM, Davide Anastasia < davide.anastasia@gmail.com> wrote: > Hi Tyler, > I am interested in this scenario as well: could you please elaborate > further your answer? > > Thanks a lot, > Davide > On 19 Jun 2013 16:01, "Tyler Hobbs" <tyler@datastax.com> wrote: > >> >> On Wed, Jun 19, 2013 at 8:08 AM, Ryan, Brent <BRyan@cvent.com> wrote: >> >>> >>> CREATE TABLE count3 ( >>> counter text, >>> ts timeuuid, >>> key1 text, >>> value int, >>> PRIMARY KEY ((counter, ts)) >>> ) >>> >> >> Instead of doing a composite partition key, remove a set of parens and >> let ts be your clustering key. That will cause cql rows to be stored in >> sorted order by the ts column (for a given value of "counter") and allow >> you to do the kind of query you're looking for. >> >> >> -- >> Tyler Hobbs >> DataStax <http://datastax.com/> >> >
Hi Tyler, I am interested in this scenario as well: could you please elaborate further your answer? Thanks a lot, Davide On 19 Jun 2013 16:01, "Tyler Hobbs" <tyler@datastax.com> wrote: > > On Wed, Jun 19, 2013 at 8:08 AM, Ryan, Brent <BRyan@cvent.com> wrote: > >> >> CREATE TABLE count3 ( >> counter text, >> ts timeuuid, >> key1 text, >> value int, >> PRIMARY KEY ((counter, ts)) >> ) >> > > Instead of doing a composite partition key, remove a set of parens and let > ts be your clustering key. That will cause cql rows to be stored in sorted > order by the ts column (for a given value of "counter") and allow you to do > the kind of query you're looking for. > > > -- > Tyler Hobbs > DataStax <http://datastax.com/> >
On Wed, Jun 19, 2013 at 7:47 AM, Ben Boule <Ben_Boule@rapid7.com> wrote:
> Can anyone explain this to me? I have been looking through the source
> code but can't seem to find the answer.
>
> The documentation mentions using the token() function to change a value
> into it's token for use in queries. It always mentions it as taking a
> single parameter:
>
> SELECT * FROM posts WHERE token(userid) > token('tom') AND token(userid) < token('bob')
>
>
> However on my 1.2.5 node I am getting the following error:
>
> e.x.
>
> create table foo (
> organization text,
> type text,
> time timestamp,
> id uuid,
> primary key ((organization, type, time), id))
>
> select * from foo where organization = 'companyA' and type = 'typeB' and
> token(time) < token('somevalue') and token(time) > token('othervalue')
>
> Bad Request: Invalid number of arguments in call to function token: 3
> required but 1 provided
>
> What are the other two parameters? We don't currently use the token
> function but I was experimenting seeing if I could move the time into the
> partition key for a table like this to better distribute the rows. But I
> can't seem to figure out how to get token() working.
>
token() acts on the entire partition key, which for you is (organization,
type, time), hence the 3 required values.
In order to better distribute the rows, I suggest using a time bucket as
part of the partition key. For example, you might use only the date
portion of the timestamp as the time bucket.
These posts talk about doing something similar with the Thrift API, but
they will probably still be helpful:
- http://rubyscale.com/2011/basic-time-series-with-cassandra/
- http://www.datastax.com/dev/blog/advanced-time-series-with-cassandra
--
Tyler Hobbs
DataStax <http://datastax.com/>
On Wed, Jun 19, 2013 at 8:08 AM, Ryan, Brent <BRyan@cvent.com> wrote: > > CREATE TABLE count3 ( > counter text, > ts timeuuid, > key1 text, > value int, > PRIMARY KEY ((counter, ts)) > ) > Instead of doing a composite partition key, remove a set of parens and let ts be your clustering key. That will cause cql rows to be stored in sorted order by the ts column (for a given value of "counter") and allow you to do the kind of query you're looking for. -- Tyler Hobbs DataStax <http://datastax.com/>
Hi, Our Hadoop jobs will only do READs and we want to restrict reads in this dedicated DC even if performances are bad. What can we do to achieve this goal ? - set dynamic_snitch_badness_threshold to 0.98 on these DC's nodes ? can we have different dynamic_snitch_badness_threshold values on nodes from different DC ? - for consistency to LOCAL_QUORUM ? However if we have more than one replica we'll do more reads thanks -- Cyril SCETBON _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified. Thank you.
On Tue, Jun 18, 2013 at 4:04 AM, aaron morton <aaron@thelastpickle.com> wrote: >> Even more if we could automate some up-scale thanks to AWS alarms, It >> would be awesome. > > I saw a demo for Priam (https://github.com/Netflix/Priam) doing that at > netflix in March, not sure if it's public yet. > >> Are the vnodes feature and the tokens =>vnodes transition safe enough to >> go live with vnodes ? > > There have been some issues, search the user list for shuffle and as always > test. > >> Any advice about vnodes ? > > They are in use out there. It's a sizable change so it would be good idea to > build a test system for running shuffle and testing your application. There > have been some issues with repair and range scans (including hadoop > integration.) Also, in his presentation at last week's Summit, Eric Evans suggested not using shuffle. As an alternative he suggested removing and replacing nodes one-by-one. Jim > > Cheers > > ----------------- > Aaron Morton > Freelance Cassandra Consultant > New Zealand > > @aaronmorton > http://www.thelastpickle.com > > On 18/06/2013, at 7:04 PM, Alain RODRIGUEZ <arodrime@gmail.com> wrote: > > Any insights on vnodes, one month after my original post ? > > > 2013/5/16 Alain RODRIGUEZ <arodrime@gmail.com> >> >> Hi, >> >> Adding vnodes is a big improvement to Cassandra, specifically because we >> have a fluctuating load on our Cassandra depending on the week, and it is >> quite annoying to add some nodes for one week or two, move tokens and then >> having to remove them and then move tokens again. Even more if we could >> automate some up-scale thanks to AWS alarms, It would be awesome. >> >> We don't use vnodes yet because Opscenter did not support this feature and >> because we need to have a reliable production. Now Opscenter handles vnodes. >> >> Are the vnodes feature and the tokens =>vnodes transition safe enough to >> go live with vnodes ? >> >> What would be the transition process ? >> >> Does someone auto-scale his Cassandra cluster ? >> >> Any advice about vnodes ? > > >
You really do not need much in java you can use the embedded server. Hector wrap a simple class around thiscalled EmbeddedServerHelper On Wednesday, June 19, 2013, Ben Boule <Ben_Boule@rapid7.com> wrote: > Hi Shabab, > > Cassandra-Unit has been helpful for us for running unit tests without requiring a real cassandra instance to be running. We only use this to test our "DAO" code which interacts with the Cassandra client. It basically starts up an embedded instance of cassandra and fools your client/driver into using it. It uses a non-standard port and you just need to make sure you can set the port as a parameter into your client code. > > https://github.com/jsevellec/cassandra-unit > > One important thing is to either clear out the keyspace in between tests or carefully separate your data so different tests don't collide with each other in the embedded database. > > Setup/tear down time is pretty reasonable. > > Ben > ________________________________ > From: Shahab Yunus [shahab.yunus@gmail.com] > Sent: Wednesday, June 19, 2013 8:46 AM > To: user@cassandra.apache.org > Subject: Re: Unit Testing Cassandra > > Thanks Stephen for you reply and explanation. My bad that I mixed those up and wasn't clear enough. Yes, I have different 2 requests/questions. > 1) One is for the unit testing. > 2) Second (in which I am more interested in) is for performance (stress/load) testing. Let us keep integration aside for now. > I do see some stuff out there but wanted to know recommendations from the community given their experience. > Regards, > Shahab > > On Wed, Jun 19, 2013 at 3:15 AM, Stephen Connolly < stephen.alan.connolly@gmail.com> wrote: >> >> Unit testing means testing in isolation the smallest part. >> Unit tests should not take more than a few milliseconds to set up and verify their assertions. >> As such, if your code is not factored well for testing, you would typically use mocking (either by hand, or with mocking libraries) to mock out the bits not under test. >> Extensive use of mocks is usually a smell of code that is not well designed *for testing* >> If you intend to test components integrated together... That is integration testing. >> If you intend to test performance of the whole or significant parts of the whole... That is performance testing. >> When searching for the above, you will not get much luck if you are looking for them in the context of "unit testing" as those things are *outside the scope of unit testing" >> >> On Wednesday, 19 June 2013, Shahab Yunus wrote: >>> >>> Hello, >>> >>> Can anyone suggest a good/popular Unit Test tools/frameworks/utilities out >>> there for unit testing Cassandra stores? I am looking for testing from performance/load and monitoring perspective. I am using 1.2. >>> >>> Thanks a lot. >>> >>> Regards, >>> Shahab >> >> >> -- >> Sent from my phone > > This electronic message contains information which may be confidential or privileged. The information is intended for the use of the individual or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic transmission in error, please notify us by e-mail at (postmaster@rapid7.com) immediately.
I'm experimenting with a data model that will need to ingest a lot of data that will need to
be query able by time. In the example below, I want to be able to run a query like "select
* from count3 where counter = 'test' and ts > minTimeuuid('2013-06-18 16:23:00') and ts
< minTimeuuid('2013-06-18 16:24:00');". However, in certain cases this query fails with
the error "Bad Request: Start key must sort before (or equal to) finish key in your partitioner!".
It's not clear to be why this happens or what the issue is as it seems like a bug.
Here's the table:
CREATE TABLE count3 (
counter text,
ts timeuuid,
key1 text,
value int,
PRIMARY KEY ((counter, ts))
)
It has data like so:
cqlsh:Statistics> select counter,dateof(ts),key1,value from count3;
counter | dateof(ts) | key1 | value
---------+--------------------------+------+-------
test | 2013-06-18 16:23:25-0400 | 1 | 1
test | 2013-06-18 16:23:28-0400 | 1 | 1
test | 2013-06-18 16:23:28-0400 | 1 | 1
test | 2013-06-18 16:23:28-0400 | 1 | 1
test | 2013-06-18 16:23:29-0400 | 1 | 1
test | 2013-06-18 16:23:29-0400 | 1 | 1
test | 2013-06-18 16:23:29-0400 | 1 | 1
test | 2013-06-18 16:23:30-0400 | 1 | 1
test | 2013-06-18 16:23:30-0400 | 1 | 1
test | 2013-06-18 16:23:31-0400 | 1 | 1
test | 2013-06-18 16:23:31-0400 | 1 | 1
test | 2013-06-18 16:23:31-0400 | 1 | 1
test | 2013-06-18 16:23:32-0400 | 1 | 1
test | 2013-06-18 16:23:32-0400 | 1 | 1
DOESN'T WORK:
cqlsh:Statistics> select * from count3 where counter = 'test' and ts > minTimeuuid('2013-06-18
16:23:00') and ts < minTimeuuid('2013-06-18 16:24:00');
Bad Request: Start key must sort before (or equal to) finish key in your partitioner!
WORKS FINE:
cqlsh:Statistics> select * from count3 where counter = 'test' and ts > minTimeuuid('2013-06-18
16:23:25') and ts < minTimeuuid('2013-06-18 16:23:31');
counter | ts | key1 | value
---------+--------------------------------------+------+-------
test | edee0df0-d854-11e2-ac46-cba9e55f995d | 1 | 1
test | ef9a5e60-d854-11e2-ac46-cba9e55f995d | 1 | 1
test | efccb900-d854-11e2-ac46-cba9e55f995d | 1 | 1
test | effb1c00-d854-11e2-ac46-cba9e55f995d | 1 | 1
test | f0284680-d854-11e2-ac46-cba9e55f995d | 1 | 1
test | f05b8b80-d854-11e2-ac46-cba9e55f995d | 1 | 1
test | f08c5f80-d854-11e2-ac46-cba9e55f995d | 1 | 1
test | f0c6f780-d854-11e2-ac46-cba9e55f995d | 1 | 1
test | f1018f80-d854-11e2-ac46-cba9e55f995d | 1 | 1
Thanks,
Brent