Return-Path: 
 <user-return-6212-apmail-storm-user-archive=storm.apache.org@storm.apache.org>
X-Original-To: apmail-storm-user-archive@minotaur.apache.org
Delivered-To: apmail-storm-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 4E8FC10CCD
	for <apmail-storm-user-archive@minotaur.apache.org>;
 Sun, 15 Feb 2015 12:42:50 +0000 (UTC)
Received: (qmail 5159 invoked by uid 500); 15 Feb 2015 12:42:49 -0000
Delivered-To: apmail-storm-user-archive@storm.apache.org
Received: (qmail 5104 invoked by uid 500); 15 Feb 2015 12:42:49 -0000
Mailing-List: contact user-help@storm.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@storm.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@storm.apache.org>
List-Post: <mailto:user@storm.apache.org>
List-Id: <user.storm.apache.org>
Reply-To: user@storm.apache.org
Delivered-To: mailing list user@storm.apache.org
Received: (qmail 5094 invoked by uid 99); 15 Feb 2015 12:42:49 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 15 Feb 2015 12:42:49 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of clearmidoubt@gmail.com
 designates 209.85.213.179 as permitted sender)
Received: from [209.85.213.179] (HELO mail-ig0-f179.google.com)
 (209.85.213.179)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 15 Feb 2015 12:42:23 +0000
Received: by mail-ig0-f179.google.com with SMTP id l13so18951129iga.0
        for <user@storm.apache.org>; Sun, 15 Feb 2015 04:40:52 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:date:message-id:subject:from:to:content-type;
        bh=PIHVuOWC44vqGwC7Ea0tkoncqi05cER9WKABKTvRZGA=;
        b=EvQoWypCBK/qRZCGx5hgmYw00mSzcQySb0n/LMJblOa1XzReF5yqbgQXRwF8S//ThR
         cgwUz6TQfAX3UHsTn3x4KsiN//QiYNMcJ2t2Gem2GP2IiUQvZI/JkDdPpUS3eCRDJG4Y
         ichhn39BUzSO1wzjMv1pkcBO3m2NLhXteKrrlMaPU3g/E/JwavVYVrjCR40lDtOpOU0p
         MR+ETs6Lgqiz8eczsZarpfUu3FRAMzc+PqBjrpmmeRWtJmyi9OVsNb94Wm4yxXxCXcz2
         +TbQfqhS3JXF8/Wv9ht5IKbobS6Wy3xNlldxZ2IrpGb+8ImJSky1WLbSWhrkqKiuJVO1
         U+og==
MIME-Version: 1.0
X-Received: by 10.50.85.44 with SMTP id e12mr15852763igz.48.1424004051999;
 Sun, 15 Feb 2015 04:40:51 -0800 (PST)
Received: by 10.36.73.85 with HTTP; Sun, 15 Feb 2015 04:40:51 -0800 (PST)
Date: Sun, 15 Feb 2015 18:10:51 +0530
Message-ID: 
 <CAOi=ZKRPY3_aN4YfkptbeJXbkFOtZiN5-4bq4CpFm_g+cjN=dg@mail.gmail.com>
Subject: Number of Events Mismatch from source to destination
From: Vineet Mishra <clearmidoubt@gmail.com>
To: users@kafka.apache.org, user@storm.apache.org
Content-Type: multipart/alternative; boundary=089e01536d1aada130050f1fc74f
X-Virus-Checked: Checked by ClamAV on apache.org

--089e01536d1aada130050f1fc74f
Content-Type: text/plain; charset=UTF-8

Hi All,

I am having a Kafka Storm Topology which is ingesting events published to
Kafka and processing on top of that data.

Although apart from some latency I found that everything was going good.
But recently I came across a issue which I couldn't get any solution yet.

I publishing some events from Logstash to Kafka and which is being
Subscribed by Storm Topology for further processing, I could see that the
source record count and the events processed by Storm is varying by a
reasonable number. So I have around 200 Million events to be processed out
of which 10 Million Events are getting lost as I could see the
acknowledgement of 190 Million events in the Storm.

Stuck at this issue, looking for expert advise.

Thanks!

--089e01536d1aada130050f1fc74f
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi All,<div><br></div><div>I am having a Kafka Storm Topol=
ogy which is ingesting events published to Kafka and processing on top of t=
hat data.</div><div><br></div><div>Although apart from some latency I found=
 that everything was going good. But recently I came across a issue which I=
 couldn&#39;t get any solution yet.</div><div><br></div><div>I publishing s=
ome events from Logstash to Kafka and which is being Subscribed by Storm To=
pology for further processing, I could see that the source record count and=
 the events processed by Storm is varying by a reasonable number. So I have=
 around 200 Million events to be processed out of which 10 Million Events a=
re getting lost as I could see the acknowledgement of 190 Million events in=
 the Storm.</div><div><br></div><div>Stuck at this issue, looking for exper=
t advise.</div><div><br></div><div>Thanks!</div></div>

--089e01536d1aada130050f1fc74f--