Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 68373 invoked from network); 21 Aug 2010 00:35:06 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 21 Aug 2010 00:35:06 -0000 Received: (qmail 73161 invoked by uid 500); 21 Aug 2010 00:35:05 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 73102 invoked by uid 500); 21 Aug 2010 00:35:04 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 73094 invoked by uid 99); 21 Aug 2010 00:35:04 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 21 Aug 2010 00:35:04 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=10.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ronald.park@cbsinteractive.com designates 74.125.149.199 as permitted sender) Received: from [74.125.149.199] (HELO na3sys009aog108.obsmtp.com) (74.125.149.199) by apache.org (qpsmtpd/0.29) with SMTP; Sat, 21 Aug 2010 00:34:58 +0000 Received: from source ([209.85.216.46]) by na3sys009aob108.postini.com ([74.125.148.12]) with SMTP ID DSNKTG8fHebuqoeQuoauN/Ie4w1QSBR0ASvo@postini.com; Fri, 20 Aug 2010 17:34:38 PDT Received: by qwh6 with SMTP id 6so4126933qwh.33 for ; Fri, 20 Aug 2010 17:34:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.235.66 with SMTP id kf2mr1678739qcb.2.1282350877033; Fri, 20 Aug 2010 17:34:37 -0700 (PDT) Received: by 10.229.75.82 with HTTP; Fri, 20 Aug 2010 17:34:36 -0700 (PDT) In-Reply-To: References: Date: Fri, 20 Aug 2010 20:34:36 -0400 Message-ID: Subject: Re: "MessageDeserializationTask.java (line 47) dropping message" errors From: Ronald Park To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=0016e64bf1ee80c47b048e4a945a --0016e64bf1ee80c47b048e4a945a Content-Type: text/plain; charset=ISO-8859-1 Oh, excellent. Ok, my code does some retrying (including some sleep then retry logic) when I get a TimedOut, so I think I'm actually just fine. Whew. :) I had just noticed the dropped message in Cassandra's log today but had seen the time outs in my log while it was running. I just didn't connect the two together. Thanks again, Ron On Fri, Aug 20, 2010 at 6:26 PM, Jonathan Ellis wrote: > On Fri, Aug 20, 2010 at 5:02 PM, Ronald Park > wrote: > > Well this is bad news... Why is this logged only as a WARN? Seems quite > > SEVERE to me. > > That's as high as log4j goes, without going all the way to ERROR which > we reserve for bugs. (High load is not a bug.) > > > Fortunately for my app, I can go back to my source, recreate the data and > > reload it at a slower rate (and cross my fingers that this rate is 'slow > > enough')... unfortunately, it took me 20 hours to load the data (200M > > records) at a rate of 3K/sec. It looks like I only dropped about 600K > > records out of 200M (.3%) ... but no way to tell which ones were dropped. > :( > > Sure there is. Your client will have gotten a TimedOutException if > Cassandra wasn't able to meet your requested ConsistencyLevel. And if > it was, then no problem. > --0016e64bf1ee80c47b048e4a945a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Oh, excellent. =A0Ok, my code does some retrying (including some sleep then= retry logic) when I get a TimedOut, so I think I'm actually just fine.= =A0Whew. :) =A0 I had just noticed the dropped message in Cassandra's = log today but had seen the time outs in my log while it was running. I just= didn't connect the two together.

Thanks again,
Ron

On Fri, Aug 20, 2010 at 6:26 PM, Jonathan Ellis <jbellis@gmail.com> wrote:<= br>
On Fri, Aug 20, 2010 at 5:02 PM, Ronald Par= k
<ron= ald.park@cbsinteractive.com> wrote:
> Well this is bad news... Why is this logged on= ly as a WARN?=A0 Seems quite
> SEVERE to me.

That's as high as log4j goes, without going all the way to ERROR = which
we reserve for bugs. =A0(High load is not a bug.)

> Fortunately for my app, I can go back to my source, recreate the data = and
> reload it at a slower rate (and cross my fingers that this rate is = 9;slow
> enough')... unfortunately, it took me 20 hours to load the data (2= 00M
> records) at a rate of 3K/sec.=A0 It looks like I only dropped about 60= 0K
> records out of 200M (.3%) ... but no way to tell which ones were dropp= ed. :(

Sure there is. =A0Your client will have gotten a TimedOutException if=
Cassandra wasn't able to meet your requested ConsistencyLevel. =A0And i= f
it was, then no problem.

--0016e64bf1ee80c47b048e4a945a--