incubator-cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Burruss <bburr...@expedia.com>
Subject Re: hinted handoff - 1.1.0 beta1
Date Tue, 21 Feb 2012 05:10:05 GMT
https://issues.apache.org/jira/browse/CASSANDRA-3935


On 2/20/12 7:07 PM, "Jonathan Ellis" <jbellis@gmail.com> wrote:

>Hmm, that dates back to early 0.7...  I think the intent is to skip
>deleted hints, even if the tombstoned hints haven't expired yet (and
>thus would not be elided by the getCF filter).  Sounds like that's
>breaking pagination.  Normally, all hints get delivered at once and
>then we compact away the tombstones -- that's my guess as to why we
>haven't hit this before.
>
>Can you create a ticket?
>
>On Mon, Feb 20, 2012 at 4:52 PM, Todd Burruss <bburruss@expedia.com>
>wrote:
>> I'm testing hinted handoff in 1.1 beta1 and cannot seem to get a hint
>>delivered.  3 node cluster, RF = 3, writing with CL = ONE.  killed a
>>host then did the write using the CLI on another node.  I can see hint
>>waiting using CLI and I see the log messages at the end of this email.
>>this suggests the hints exist bu are not being delivered (and I'll see
>>the log messages over and over.)
>>
>> I did see tracing with debugger and see that in
>>HintedHandoffManager.deliverHintsToEndpointInternal, this line will
>>remove the hint because of the Integer.MAX_VALUE
>>
>>
>>            ColumnFamily hintsPage =
>>ColumnFamilyStore.removeDeleted(hintStore.getColumnFamily(filter),
>>Integer.MAX_VALUE);
>>
>> I'm not sure I quite understand while MAX is used when the same
>>"remove" is done in getColumnFamily(filter).  regardless if it is useful
>>or not, it prevents the hints from delivery.
>>
>> any thoughts?
>>
>>
>> [default@unknown] use system;
>> Authenticated to keyspace: system
>> [default@system] list hintscolumnfamily;
>> Using default limit of 100
>> -------------------
>> RowKey: 00
>> => (super_column=493ecfa05c1411e100000da23097c7ff,
>>     (column=6b6579, value=6b35, timestamp=1329777799580, ttl=86400)
>>     (column=6d75746174696f6e,
>>value=000662746f64646200026b3500000001000003e801000003e87fffffff800000000
>>00000000000000100026335000004b96d055fd1300000000c68696e746564207772697465
>>, timestamp=1329777799579, ttl=86400)
>>     (column=7461626c65, value=62746f646462, timestamp=1329777799580,
>>ttl=86400)
>>     (column=76657273696f6e, value=00000004, timestamp=1329777799580,
>>ttl=86400))
>>
>> 1 Row Returned.
>> Elapsed time: 58 msec(s).
>>
>>
>>  INFO [HintedHandoff:1] 2012-02-20 14:44:53,811
>>HintedHandOffManager.java (line 296) Started hinted handoff for token: 0
>>with IP: /192.168.56.1
>>  INFO [HintedHandoff:1] 2012-02-20 14:44:53,815
>>HintedHandOffManager.java (line 373) Finished hinted handoff of 0 rows
>>to endpoint /192.168.56.1
>>
>>
>> I have a hint waiting to be deli
>
>
>
>-- 
>Jonathan Ellis
>Project Chair, Apache Cassandra
>co-founder of DataStax, the source for professional Cassandra support
>http://www.datastax.com


Mime
View raw message