cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: Finding big rows
Date Thu, 12 May 2011 01:47:12 GMT
Let me know if you get anywhere, I'm on there as aaron_morton but I'm also way over in New
Zealand. 

If you are using your own client and writing data you cannot read back check that the byte
encoding is always the same and that you are setting appropriate timestamps for every call.
In the log message all the columns have a 0 time stamp. Is that deliberate ? 

Does the count in  "collecting 0 of 1000000" increase ? If it does it means the query is collecting
rows that a "live" and match the query filer. If not it means all that data under a (probably
row level) tombstone. Which make me ask is this a row that gets a lot of inserts and then
deletes ?    

Cheers

-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 11 May 2011, at 23:30, Meler Wojciech wrote:

> I didn’t run nodetool scrub. My app uses c++ thrift client (0.5.0 and 0.6.1) .
> As this is production environment I get a lot of messages "collecting %s of %s", but
there is no row key.
> I’ve matched it by uuid and thread – hope it is ok:
>  
> [ReadStage:3][org.apache.cassandra.db.filter.SliceQueryFilter] collecting 0 of 1000000:
SuperColumn(455470c2-60e6-11e0-acae-e41f13798a50 [4554853a-60e6-11e0-9342-e41f13798a50:false:361745@0,aa92b386-60e6-11e0-bc58-e41f13798a50:false:159@0,ac346cc0-60e6-11e0-bf8e-e41f13798a50:false:53@0,ad16d362-60e6-11e0-ad9c-e41f13798a50:false:66@0,ae57076a-60e6-11e0-8982-e41f13798a50:false:48@0,afc042ba-60e6-11e0-8320-e41f13798a50:false:63@0,b38d912c-60e6-11e0-b3b7-e41f13798a50:false:164@0,49d7dc00-60e7-11e0-9694-e41f13798a50:false:100@0,94e99fa8-60e7-11e0-b621-e41f13798a50:false:233@0,9612c3a0-60e7-11e0-8292-e41f13798a50:false:4049@0,ec245880-60e7-11e0-85be-e41f13798a50:false:148@0,110ac968-60e8-11e0-b325-e41f13798a50:false:125@0,64a45b4c-60e9-11e0-9628-e41f13798a50:false:160@0,cfc39a00-60e9-11e0-9539-e41f13798a50:false:105@0,2a21ab22-60ea-11e0-a1f6-e41f13798a50:false:146@0,95f2d2d6-60ea-11e0-b763-e41f13798a50:false:53@0,97972362-60ea-11e0-9275-e41f13798a50:false:134@0,ce980606-60ea-11e0-bd03-e41f13798a50:false:195@0,517e02dc-60eb-11e0-b0a8-e41f13798a50:false:53@0,5694c74c-60eb-11e0-941f-e41f13798a50:false:170@0,8d48cca2-60eb-11e0-bdbc-e41f13798a50:false:187@0,fc5e0148-60eb-11e0-ac4c-e41f13798a50:false:558@0,fc7e9476-60eb-11e0-bbfe-e41f13798a50:false:161@0,22a860a0-60ec-11e0-8d1b-e41f13798a50:false:138@0,bf054b52-60ec-11e0-a4b1-e41f13798a50:false:56@0,fd3b4822-60ec-11e0-8612-e41f13798a50:false:234@0,0c1d6fe0-60ee-11e0-8bb0-e41f13798a50:false:79@0,43bbddec-60ee-11e0-848f-e41f13798a50:false:79@0,ec50ed8e-60ef-11e0-a8c1-e41f13798a50:false:60@0,57ae0534-60f1-11e0-8fd5-e41f13798a50:false:60@0,81d7bbe8-60f1-11e0-9586-e41f13798a50:false:143@0,08ce8852-60f2-11e0-bfeb-e41f13798a50:false:150@0,c51bf170-60f2-11e0-8a60-e41f13798a50:false:192@0,e98b181e-60f3-11e0-b20b-e41f13798a50:false:108@0,ed931d76-60f3-11e0-90cf-e41f13798a50:false:146@0,f754541a-60f3-11e0-84db-e41f13798a50:false:60@0,24a73220-60f4-11e0-ae03-e41f13798a50:false:139@0,33485db8-60f4-11e0-9f86-e41f13798a50:false:196@0,42fe380e-60f4-11e0-9f0f-e41f13798a50:false:150@0,440afbb0-60f4-11e0-a066-e41f13798a50:false:122@0,64d9d13a-60f5-11e0-a30c-e41f13798a50:false:60@0,9e1cefc2-60f5-11e0-826e-e41f13798a50:false:205@0,1f642a46-60f6-11e0-8aa3-e41f13798a50:false:298@0,1f6c946a-60f6-11e0-9eab-e41f13798a50:false:117@0,d1a4e8c6-60f6-11e0-9bfa-e41f13798a50:false:58@0,0a40ba5c-60f7-11e0-893f-e41f13798a50:false:170@0,a2e0f740-60f7-11e0-93b0-e41f13798a50:false:108@0,3922060e-60f8-11e0-b850-e41f13798a50:false:147@0,3cdcdf08-60f8-11e0-8320-e41f13798a50:false:60@0,79b33a26-60f8-11e0-b151-e41f13798a50:false:187@0,aae281c8-60f9-11e0-9d14-e41f13798a50:false:60@0,e1b3295a-60f9-11e0-b367-e41f13798a50:false:81@0,8149870c-60fa-11e0-b6d4-e41f13798a50:false:128@0,b483680e-60fa-11e0-a56f-e41f13798a50:false:164@0,3dfdfe78-60fb-11e0-8582-e41f13798a50:false:143@0,15a9b830-60fc-11e0-be81-e41f13798a50:false:217@0,46ae8aae-60fd-11e0-908e-e41f13798a50:false:60@0,7ee58490-60fd-11e0-9072-e41f13798a50:false:81@0,bf4fba30-60ff-11e0-9b5f-e41f13798a50:false:60@0,26201952-6101-11e0-a4cb-e41f13798a50:false:60@0,8e5e995c-6102-11e0-9bb3-e41f13798a50:false:60@0,a4ef01c0-6102-11e0-9bff-e41f13798a50:false:214@0,a5034784-6102-11e0-96ee-e41f13798a50:false:133@0,ad9f3682-6102-11e0-8df5-e41f13798a50:false:172@0,e2e01b36-6102-11e0-8489-e41f13798a50:false:155@0,25561204-6103-11e0-a14b-e41f13798a50:false:155@0,4d39d2a6-6103-11e0-844a-e41f13798a50:false:152@0,74aeb4fa-6103-11e0-9f65-e41f13798a50:false:140@0,7e8802ba-6103-11e0-81f1-e41f13798a50:false:140@0,8e6532f2-6103-11e0-acd4-e41f13798a50:false:141@0,fceadee8-6103-11e0-8beb-e41f13798a50:false:313@0,2d319618-6105-11e0-9efa-e41f13798a50:false:60@0,4d4540bc-6105-11e0-96d1-e41f13798a50:false:128@0,63fcaa02-6105-11e0-ac35-e41f13798a50:false:81@0,67c70114-6105-11e0-9ab8-e41f13798a50:false:167@0,3a104306-6106-11e0-8be9-e41f13798a50:false:145@0,ce884cf4-6106-11e0-92ba-e41f13798a50:false:208@0,fe230516-6107-11e0-a9fa-e41f13798a50:false:60@0,354f86e0-6108-11e0-970f-e41f13798a50:false:81@0,9fbcaef8-6109-11e0-9b5a-e41f13798a50:false:60@0,55b4cca4-610a-11e0-a82d-e41f13798a50:false:129@0,ddc1395c-610a-11e0-9541-e41f13798a50:false:166@0,0934dec2-610b-11e0-a63b-e41f13798a50:false:192@0,39562880-610c-11e0-af41-e41f13798a50:false:310@0,74296bb6-610c-11e0-bdc4-e41f13798a50:false:192@0,cfe50906-610c-11e0-9fc9-e41f13798a50:false:108@0,dd594b9c-610c-11e0-9db1-e41f13798a50:false:60@0,dd5ce428-610c-11e0-8962-e41f13798a50:false:92@0,1a0804d4-610d-11e0-b783-e41f13798a50:false:178@0,5135e390-610d-11e0-9d71-e41f13798a50:false:62@0,a3f68ada-610d-11e0-a6ef-e41f13798a50:false:60@0,db2a37a4-610d-11e0-b795-e41f13798a50:false:62@0,15802f0c-610f-11e0-abfe-e41f13798a50:false:128@0,42d810be-610f-11e0-8f4d-e41f13798a50:false:62@0,54456e96-610f-11e0-88a0-e41f13798a50:false:165@0,7b7b9e54-610f-11e0-b1e2-e41f13798a50:false:139@0,20b637bc-6110-11e0-b107-e41f13798a50:false:180@0,746d9be8-6110-11e0-9c9a-e41f13798a50:false:60@0,af7f87be-6110-11e0-8c1b-e41f13798a50:false:62@0,50e10916-6111-11e0-9d9a-e41f13798a50:false:128@0,1815bcde-6112-11e0-900e-e41f13798a50:false:160@0,4539798a-6112-11e0-84a8-e41f13798a50:false:217@0,45451d1c-6112-11e0-a6f5-e41f13798a50:false:110@0,cb4ae90a-6112-11e0-baab-e41f13798a50:false:146@0,cd86943a-6112-11e0-893b-e41f13798a50:false:119@0,490a443a-6113-11e0-a2f5-e41f13798a50:false:60@0,8cbfb584-6113-11e0-91d3-e41f13798a50:false:190@0,15215c34-6114-11e0-aec4-e41f13798a50:false:108@0,bdd0c716-6114-11e0-b8b0-e41f13798a50:false:60@0,f4e4446c-6114-11e0-a772-e41f13798a50:false:165@0,26b972d6-6116-11e0-bb90-e41f13798a50:false:60@0,7814059c-6116-11e0-bac6-e41f13798a50:false:62@0,94e63e9c-6116-11e0-b28b-e41f13798a50:false:127@0,e27c6a68-6117-11e0-b4b3-e41f13798a50:false:164@0,131754e8-6119-11e0-a9b0-e41f13798a50:false:60@0,4b32a4d6-6119-11e0-adda-e41f13798a50:false:62@0,7df5d5d6-611a-11e0-be7e-e41f13798a50:false:62@0,6012816c-611b-11e0-9795-e41f13798a50:false:128@0,620fab98-611b-11e0-b2e0-e41f13798a50:false:164@0,6325e7d6-611b-11e0-802a-e41f13798a50:false:139@0,65258b9a-611b-11e0-b598-e41f13798a50:false:154@0,655f9146-611b-11e0-ac19-e41f13798a50:false:155@0,67c337c6-611b-11e0-bc49-e41f13798a50:false:144@0,e8e8ff48-611b-11e0-be42-e41f13798a50:false:253@0,610ad578-611c-11e0-b7f7-e41f13798a50:false:242@0,6120cb1c-611c-11e0-9308-e41f13798a50:false:135@0,75bbf9c0-611c-11e0-b8b3-e41f13798a50:false:148@0,19ed7636-611d-11e0-adc5-e41f13798a50:false:61@0,515b2078-611d-11e0-afa7-e41f13798a50:false:178@0,9e68ba2e-611d-11e0-9fca-e41f13798a50:false:110@0,82113080-611e-11e0-9aee-e41f13798a50:false:61@0,bcbfb760-611e-11e0-bee8-e41f13798a50:false:165@0,1b236b44-611f-11e0-9f97-e41f13798a50:false:110@0,35f553b0-611f-11e0-a0c2-e41f13798a50:false:61@0,5c9072fc-611f-11e0-b81c-e41f13798a50:false:166@0,70a9b898-611f-11e0-a4c4-e41f13798a50:false:173@0,432459c0-6122-11e0-b93a-e41f13798a50:false:147@0,8a11c9d0-6122-11e0-9ae7-e41f13798a50:false:123@0,8a3081e0-6122-11e0-949a-e41f13798a50:false:137@0,b6ca7206-6122-11e0-9b27-e41f13798a50:false:125@0,c30ae4c4-6122-11e0-b50e-e41f13798a50:false:138@0,79f580fe-6123-11e0-ac09-e41f13798a50:false:120@0,bb123550-6123-11e0-8eaf-e41f13798a50:false:137@0,bca00ad2-6123-11e0-87f1-e41f13798a50:false:120@0,c34c7730-6123-11e0-9d3c-e41f13798a50:false:137@0,dc6de510-6127-11e0-b924-e41f13798a50:false:120@0,aa9c885a-6129-11e0-9bb0-e41f13798a50:false:136@0,30aa16e2-612a-11e0-9823-e41f13798a50:false:120@0,5911474e-612b-11e0-a19e-e41f13798a50:false:137@0,59acbcf6-612b-11e0-890d-e41f13798a50:false:120@0,5a4d3960-612b-11e0-9f6e-e41f13798a50:false:136@0,5af0d048-612b-11e0-81ef-e41f13798a50:false:120@0,5cbc91e6-612b-11e0-85a5-e41f13798a50:false:137@0,9b09d3fa-612b-11e0-86d7-e41f13798a50:false:120@0,9bd9527e-612b-11e0-984c-e41f13798a50:false:137@0,6d7436d6-612d-11e0-a7eb-e41f13798a50:false:119@0,8e928714-612d-11e0-910f-e41f13798a50:false:137@0,04db8efc-612e-11e0-bd08-e41f13798a50:false:120@0,092da8be-612e-11e0-96d1-e41f13798a50:false:137@0,0c471170-612e-11e0-9fab-e41f13798a50:false:120@0,0e2f2fa4-612e-11e0-a03d-e41f13798a50:false:137@0,116bb444-612e-11e0-96b0-e41f13798a50:false:120@0,152c9a4e-612e-11e0-a302-e41f13798a50:false:137@0,18d207f6-612e-11e0-
>  
> Such msg repeats 2 times after „LocalReadRunnable reading SliceFromReadCommand” with
the big row key.
>  
> Switching to IRC…
>  
> Regards,
> Wojtek
>  
> From: aaron morton [mailto:aaron@thelastpickle.com] 
> Sent: Wednesday, May 11, 2011 12:20 PM
> To: user@cassandra.apache.org
> Subject: Re: Finding big rows
>  
> Couple of questions to ask. You may also get some value from the #cassandra chat room
where you can have a bit more of a conversation. 
>  
> - checking you ran  nodetool scrub when upgrading to 0.7.3 ? (not related to the current
problem, just asking)
> - what client library was using to write the data ?
> - when you have DEBUG logging and run the get that fails do you see any  log messages
that say "collecting %s of %s" ? (these mean the columns are been read by the query even if
not returned). 
> - not sure how easy it's going to be to pull 400MB of data through the server in one
call. Take a look at thrift_max_message_length_in_mb and thrift_framed_transport_size_in_mb
in the config. 
>  
> Hope that helps. 
>  
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
> 
> 
> "WIRTUALNA POLSKA" Spolka Akcyjna z siedziba w Gdansku przy ul. Traugutta 115 C, wpisana
do Krajowego Rejestru Sadowego - Rejestru Przedsiebiorcow prowadzonego przez Sad Rejonowy
Gdansk - Polnoc w Gdansku pod numerem KRS 0000068548, o kapitale zakladowym 67.980.024,00
zlotych oplaconym w calosci oraz Numerze Identyfikacji Podatkowej 957-07-51-216.
> 


Mime
View raw message