Mailing-List: contact dev-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of jbellis@gmail.com designates
 209.85.214.44 as permitted sender)
Received-SPF: pass (google.com: domain of jbellis@gmail.com designates
 10.204.10.91 as permitted sender) client-ip=10.204.10.91;
MIME-Version: 1.0
In-Reply-To: <CB680EB2.4856%bburruss@expedia.com>
References: <CB680EB2.4856%bburruss@expedia.com>
From: Jonathan Ellis <jbellis@gmail.com>
Date: Mon, 20 Feb 2012 21:07:35 -0600
Message-ID: 
 <CALdd-zgy0h0zZFSzE+cYwRrgqGPVeywshYKGZVC1HqZ78k53Ug@mail.gmail.com>
Subject: Re: hinted handoff - 1.1.0 beta1
To: dev@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hmm, that dates back to early 0.7...  I think the intent is to skip
deleted hints, even if the tombstoned hints haven't expired yet (and
thus would not be elided by the getCF filter).  Sounds like that's
breaking pagination.  Normally, all hints get delivered at once and
then we compact away the tombstones -- that's my guess as to why we
haven't hit this before.

Can you create a ticket?

On Mon, Feb 20, 2012 at 4:52 PM, Todd Burruss <bburruss@expedia.com> wrote:
> I'm testing hinted handoff in 1.1 beta1 and cannot seem to get a hint del=
ivered. =A03 node cluster, RF =3D 3, writing with CL =3D ONE. =A0killed a h=
ost then did the write using the CLI on another node. =A0I can see hint wai=
ting using CLI and I see the log messages at the end of this email. =A0this=
 suggests the hints exist bu are not being delivered (and I'll see the log =
messages over and over.)
>
> I did see tracing with debugger and see that in HintedHandoffManager.deli=
verHintsToEndpointInternal, this line will remove the hint because of the I=
nteger.MAX_VALUE
>
>
> =A0 =A0 =A0 =A0 =A0 =A0ColumnFamily hintsPage =3D ColumnFamilyStore.remov=
eDeleted(hintStore.getColumnFamily(filter), Integer.MAX_VALUE);
>
> I'm not sure I quite understand while MAX is used when the same "remove" =
is done in getColumnFamily(filter). =A0regardless if it is useful or not, i=
t prevents the hints from delivery.
>
> any thoughts?
>
>
> [default@unknown] use system;
> Authenticated to keyspace: system
> [default@system] list hintscolumnfamily;
> Using default limit of 100
> -------------------
> RowKey: 00
> =3D> (super_column=3D493ecfa05c1411e100000da23097c7ff,
> =A0 =A0 (column=3D6b6579, value=3D6b35, timestamp=3D1329777799580, ttl=3D=
86400)
> =A0 =A0 (column=3D6d75746174696f6e, value=3D000662746f64646200026b3500000=
001000003e801000003e87fffffff80000000000000000000000100026335000004b96d055f=
d1300000000c68696e746564207772697465, timestamp=3D1329777799579, ttl=3D8640=
0)
> =A0 =A0 (column=3D7461626c65, value=3D62746f646462, timestamp=3D132977779=
9580, ttl=3D86400)
> =A0 =A0 (column=3D76657273696f6e, value=3D00000004, timestamp=3D132977779=
9580, ttl=3D86400))
>
> 1 Row Returned.
> Elapsed time: 58 msec(s).
>
>
> =A0INFO [HintedHandoff:1] 2012-02-20 14:44:53,811 HintedHandOffManager.ja=
va (line 296) Started hinted handoff for token: 0 with IP: /192.168.56.1
> =A0INFO [HintedHandoff:1] 2012-02-20 14:44:53,815 HintedHandOffManager.ja=
va (line 373) Finished hinted handoff of 0 rows to endpoint /192.168.56.1
>
>
> I have a hint waiting to be deli


--=20
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com