cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yan Chunlu <springri...@gmail.com>
Subject Re: node restart taking too long
Date Tue, 16 Aug 2011 11:35:17 GMT
 I saw alot slicequeryfilter things if changed the log level to DEBUG.  just
thought even bring up a new node will be faster than start the old one.....
it is wired

DEBUG [main] 2011-08-16 06:32:49,213 SliceQueryFilter.java (line 123)
collecting 0 of 2147483647: 76616c7565:false:225@1313068845474382
DEBUG [main] 2011-08-16 06:32:49,245 SliceQueryFilter.java (line 123)
collecting 0 of 2147483647: 76616c7565:false:453@1310999270198313
DEBUG [main] 2011-08-16 06:32:49,251 SliceQueryFilter.java (line 123)
collecting 0 of 2147483647: 76616c7565:false:26@1313199902088827
DEBUG [main] 2011-08-16 06:32:49,576 SliceQueryFilter.java (line 123)
collecting 0 of 2147483647: 76616c7565:false:157@1313097239332314
DEBUG [main] 2011-08-16 06:32:50,674 SliceQueryFilter.java (line 123)
collecting 0 of 2147483647: 76616c7565:false:41729@1313190821826229
DEBUG [main] 2011-08-16 06:32:50,811 SliceQueryFilter.java (line 123)
collecting 0 of 2147483647: 76616c7565:false:6@1313174157301203
DEBUG [main] 2011-08-16 06:32:50,867 SliceQueryFilter.java (line 123)
collecting 0 of 2147483647: 76616c7565:false:98@1312011362250907
DEBUG [main] 2011-08-16 06:32:50,881 SliceQueryFilter.java (line 123)
collecting 0 of 2147483647: 76616c7565:false:42@1313201711997005
DEBUG [main] 2011-08-16 06:32:50,910 SliceQueryFilter.java (line 123)
collecting 0 of 2147483647: 76616c7565:false:96@1312939986190155
DEBUG [main] 2011-08-16 06:32:50,954 SliceQueryFilter.java (line 123)
collecting 0 of 2147483647: 76616c7565:false:621@1313192538616112



On Tue, Aug 16, 2011 at 7:32 PM, Yan Chunlu <springrider@gmail.com> wrote:

> but it seems the row cache is cluster wide, how will  the change of row
> cache affect the read speed?
>
>
> On Mon, Aug 15, 2011 at 7:33 AM, Jonathan Ellis <jbellis@gmail.com> wrote:
>
>> Or leave row cache enabled but disable cache saving (and remove the
>> one already on disk).
>>
>> On Sun, Aug 14, 2011 at 5:05 PM, aaron morton <aaron@thelastpickle.com>
>> wrote:
>> >  INFO [main] 2011-08-14 09:24:52,198 ColumnFamilyStore.java (line 547)
>> > completed loading (1744370 ms; 200000 keys) row cache for COMMENT
>> >
>> > It's taking 29 minutes to load 200,000 rows in the  row cache. Thats a
>> > pretty big row cache, I would suggest reducing or disabling it.
>> > Background
>> http://www.datastax.com/dev/blog/maximizing-cache-benefit-with-cassandra
>> >
>> > and server can not afford the load then crashed. after come back, node 3
>> can
>> > not return for more than 96 hours
>> >
>> > Crashed how ?
>> > You may be seeing https://issues.apache.org/jira/browse/CASSANDRA-2280
>> > Watch nodetool compactionstats to see when the Merkle tree build
>> finishes
>> > and nodetool netstats to see which CF's are streaming.
>> > Cheers
>> > -----------------
>> > Aaron Morton
>> > Freelance Cassandra Developer
>> > @aaronmorton
>> > http://www.thelastpickle.com
>> > On 15 Aug 2011, at 04:23, Yan Chunlu wrote:
>> >
>> >
>> > I got 3 nodes and RF=3, when I repairing ndoe3, it seems alot data
>> > generated.  and server can not afford the load then crashed.
>> > after come back, node 3 can not return for more than 96 hours
>> >
>> > for 34GB data, the node 2 could restart and back online within 1 hour.
>> >
>> > I am not sure what's wrong with node3 and should I restart node 3 again?
>> > thanks!
>> >
>> > Address         Status State   Load            Owns    Token
>> >
>> > 113427455640312821154458202477256070484
>> > node1     Up     Normal  34.11 GB        33.33%  0
>> > node2     Up     Normal  31.44 GB        33.33%
>> > 56713727820156410577229101238628035242
>> > node3     Down   Normal  177.55 GB       33.33%
>> > 113427455640312821154458202477256070484
>> >
>> >
>> > the log shows it is still going on, not sure why it is so slow:
>> >
>> >
>> >  INFO [main] 2011-08-14 08:55:47,734 SSTableReader.java (line 154)
>> Opening
>> > /cassandra/data/COMMENT
>> >  INFO [main] 2011-08-14 08:55:47,828 ColumnFamilyStore.java (line 275)
>> > reading saved cache /cassandra/saved_caches/COMMENT-RowCache
>> >  INFO [main] 2011-08-14 09:24:52,198 ColumnFamilyStore.java (line 547)
>> > completed loading (1744370 ms; 200000 keys) row cache for COMMENT
>> >  INFO [main] 2011-08-14 09:24:52,299 ColumnFamilyStore.java (line 275)
>> > reading saved cache /cassandra/saved_caches/COMMENT-RowCache
>> >  INFO [CompactionExecutor:1] 2011-08-14 10:24:55,480 CacheWriter.java
>> (line
>> > 96) Saved COMMENT-RowCache (200000 items) in 2535 ms
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>>
>
>

Mime
View raw message