Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 730CF200CAB for ; Sun, 18 Jun 2017 11:35:03 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 71633160BCC; Sun, 18 Jun 2017 09:35:03 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 4CA35160BE3 for ; Sun, 18 Jun 2017 11:35:01 +0200 (CEST) Received: (qmail 24655 invoked by uid 500); 18 Jun 2017 09:34:55 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 24643 invoked by uid 99); 18 Jun 2017 09:34:54 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 18 Jun 2017 09:34:54 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 24560C67B7 for ; Sun, 18 Jun 2017 09:34:54 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.236 X-Spam-Level: ** X-Spam-Status: No, score=2.236 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_COMMENT_SAVED_URL=0.357, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=openbet.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id NEKcEnbaMznf for ; Sun, 18 Jun 2017 09:34:49 +0000 (UTC) Received: from mail-wm0-f51.google.com (mail-wm0-f51.google.com [74.125.82.51]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 307AA5F2A9 for ; Sun, 18 Jun 2017 09:34:49 +0000 (UTC) Received: by mail-wm0-f51.google.com with SMTP id d73so52118265wma.0 for ; Sun, 18 Jun 2017 02:34:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=openbet.com; s=google; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=vyhkFZz9bMoGjMf0izI0EF1/djxAwqcWYjHIGF0jC1U=; b=MlZjobbSLMzA4c8/ni32eAFwcDISuAz4teQh4XkyTO4a3OcUUuWyqHzrVLIw6UTpPU bOgrY7vOlrkJ4N8K6r+ifUSrM4Kcq8VYt+wMZmYQ3NIGBt9Cx/6KM7JzA4lKe4ILN71B gPX8LcfS4foK4EP9OV3/rdATu4fQ9gGzveIOs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=vyhkFZz9bMoGjMf0izI0EF1/djxAwqcWYjHIGF0jC1U=; b=My0g6DNaHLf/kYDrkE8/1bEozLhJpirEiv63w3xilr2Zs4MYHc1zt0Tea28YGsAxZE MjhrWzbUSu7dftHaocPmD6hzpq7EvCwDIIQOzd/N6Mc11Ib6KwYx8QPoGp29AdDyurPc akjtZUlelfCcpuE6VS46WviZkWdad6WAGqWslkSK+iOxXBmnorPddT97h/ejkvca5nPW 0svS5xShmpdiEORabaNDB7M4LCq5B6kIMmO5QTVlPqTOCXKctBbqUIyb+JQAMLJSXHuH e9u8y+WSKk6V21K4Wtkm4JAGMemalwaEN4NI7f9DgwHQrltWqCqyLznXj11moKfr4XRv 5FwA== X-Gm-Message-State: AKS2vOw16TCQLW2A51gc/+AJyXqjjVxUdBi+ZrEi0f+Nr6SyeqfH6VVP FNxprVX4EP3cT0kuJFi14w== X-Received: by 10.28.62.199 with SMTP id l190mr12542447wma.13.1497778488614; Sun, 18 Jun 2017 02:34:48 -0700 (PDT) Received: from [10.7.163.190] ([46.233.112.117]) by smtp.gmail.com with ESMTPSA id 18sm4747421wmt.6.2017.06.18.02.34.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 18 Jun 2017 02:34:47 -0700 (PDT) Subject: Re: Kafka Streams vs Spark Streaming : reduce by window To: dev@kafka.apache.org, Jay Kreps Cc: "users@kafka.apache.org" , "Matthias J. Sax" References: <37755C44-6443-42A3-927A-FFBA5E06C0FC@gmail.com> <5d3fb93f-d98f-5e9e-61c1-23db0f58ff83@confluent.io> <6533b359-c69d-4225-db1e-fd0049ca9c42@openbet.com> From: Michal Borowiecki Message-ID: Date: Sun, 18 Jun 2017 10:34:42 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------7B9C862AD919005A59FAF245" Content-Language: en-GB archived-at: Sun, 18 Jun 2017 09:35:03 -0000 --------------7B9C862AD919005A59FAF245 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit If confusion is the problem, then totally agree no point adding more knobs. Perhaps you're right that users don't /really/ want processing-time semantics. Just /think/ they want them until they start considering replay/catch-up scenarios. I guess people rarely think about those from the start (I sure didn't). Cheers, Michał On 16/06/17 17:54, Jay Kreps wrote: > I think the question is when do you actually /want/ processing time > semantics? There are definitely times when its safe to assume the two > are close enough that a little lossiness doesn't matter much but it is > pretty hard to make assumptions about when the processing time is and > has been hard for us to think of a use case where its actually desirable. > > I think mostly what we've seen is confusion about the core concepts: > > * stream -- immutable events that occur > * tables (including windows) -- current state of the world > > If the root problem is confusion adding knobs never makes it better. > If the root problem is we're missing important use cases that justify > the additional knobs then i think it's good to try to really > understand them. I think there could be use cases around systems that > don't take updates, example would be email, twitter, and some metrics > stores. > > One solution that would be less complexity inducing than allowing new > semantics, but might help with the use cases we need to collect, would > be to add a new operator in the DSL. Something like .freezeAfter(30, > TimeUnit.SECONDS) that collects all updates for a given window and > both emits and enforces a single output after 30 seconds after the > advancement of stream time and remembers that it is omitted, > suppressing all further output (so the output is actually a KStream). > This might or might not depend on wall clock time. Perhaps this is in > fact what you are proposing? > > -Jay > > > > On Fri, Jun 16, 2017 at 2:38 AM, Michal Borowiecki > > > wrote: > > I wonder if it's a frequent enough use case that Kafka Streams > should consider providing this out of the box - this was asked for > multiple times, right? > > Personally, I agree totally with the philosophy of "no final > aggregation", as expressed by Eno's post, but IMO that is > predicated totally on event-time semantics. > > If users want processing-time semantics then, as the docs already > point out, there is no such thing as a late-arriving record - > every record just falls in the currently open window(s), hence the > notion of final aggregation makes perfect sense, from the > usability point of view. > > The single abstraction of "stream time" proves leaky in some cases > (e.g. for punctuate method - being addressed in KIP-138). Perhaps > this is another case where processing-time semantics warrant > explicit handling in the api - but of course, only if there's > sufficient user demand for this. > > What I could imagine is a new type of time window > (ProcessingTimeWindow?), that if used in an aggregation, the > underlying processor would force the WallclockTimestampExtractor > (KAFKA-4144 enables that) and would use the system-time > punctuation (KIP-138) to send the final aggregation value once the > window has expired and could be configured to not send > intermediate updates while the window was open. > > Of course this is just a helper for the users, since they can > implement it all themselves using the low-level API, as Matthias > pointed out already. Just seems there's recurring interest in this. > > Again, this only makes sense for processing time semantics. For > event-time semantics I find the arguments for "no final > aggregation" totally convincing. > > > Cheers, > > Michał > > > On 16/06/17 00:08, Matthias J. Sax wrote: >> Hi Paolo, >> >> This SO question might help, too: >> https://stackoverflow.com/questions/38935904/how-to-send-final-kafka-streams-aggregation-result-of-a-time-windowed-ktable >> >> >> For Streams, the basic model is based on "change" and we report updates >> to the "current" result immediately reducing latency to a minimum. >> >> Last, if you say it's going to fall into the next window, you won't get >> event time semantics but you fall back processing time semantics, that >> cannot provide exact results.... >> >> If you really want to trade-off correctness version getting (late) >> updates and want to use processing time semantics, you should configure >> WallclockTimestampExtractor and implement a "update deduplication" >> operator using table.toStream().transform(). You can attached a state to >> your transformer and store all update there (ie, newer update overwrite >> older updates). Punctuations allow you to emit "final" results for >> windows for which "window end time" passed. >> >> >> -Matthias >> >> On 6/15/17 9:21 AM, Paolo Patierno wrote: >>> Hi Eno, >>> >>> >>> regarding closing window I think that it's up to the streaming application. I mean ... >>> >>> If I want something like I described, I know that a value outside my 5 seconds window will be taken into account for the next processing (in the next 5 seconds). I don't think I'm losing a record, I am ware that this record will fall in the next "processing" window. Btw I'll take a look at your article ! Thanks ! >>> >>> >>> Paolo >>> >>> >>> Paolo Patierno >>> Senior Software Engineer (IoT) @ Red Hat >>> Microsoft MVP on Windows Embedded & IoT >>> Microsoft Azure Advisor >>> >>> Twitter : @ppatierno >>> Linkedin : paolopatierno >>> >>> Blog : DevExperience >>> >>> >>> >>> ________________________________ >>> From: Eno Thereska >>> Sent: Thursday, June 15, 2017 3:57 PM >>> To:users@kafka.apache.org >>> Subject: Re: Kafka Streams vs Spark Streaming : reduce by window >>> >>> Hi Paolo, >>> >>> Yeah, so if you want fewer records, you should actually "not" disable cache. If you disable cache you'll get all the records as you described. >>> >>> About closing windows: if you close a window and a late record arrives that should have been in that window, you basically lose the ability to process that record. In Kafka Streams we are robust to that, in that we handle late arriving records. There is a comparison here for example when we compare it to other methods that depend on watermarks or triggers:https://www.confluent.io/blog/watermarks-tables-event-time-dataflow-model/ >>> >>> >>> >>> Eno >>> >>> >>>> On 15 Jun 2017, at 14:57, Paolo Patierno wrote: >>>> >>>> Hi Emo, >>>> >>>> >>>> thanks for the reply ! >>>> >>>> Regarding the cache I'm already using CACHE_MAX_BYTES_BUFFERING_CONFIG = 0 (so disabling cache). >>>> >>>> Regarding the interactive query API (I'll take a look) it means that it's up to the application doing something like we have oob with Spark. >>>> >>>> May I ask what do you mean with "We don’t believe in closing windows" ? Isn't it much more code that user has to write for having the same result ? >>>> >>>> I'm exploring Kafka Streams and it's very powerful imho even because the usage is pretty simple but this scenario could have a lack against Spark. >>>> >>>> >>>> Thanks, >>>> >>>> Paolo. >>>> >>>> >>>> Paolo Patierno >>>> Senior Software Engineer (IoT) @ Red Hat >>>> Microsoft MVP on Windows Embedded & IoT >>>> Microsoft Azure Advisor >>>> >>>> Twitter : @ppatierno >>>> Linkedin : paolopatierno >>>> >>>> Blog : DevExperience >>>> >>>> >>>> >>>> ________________________________ >>>> From: Eno Thereska >>>> Sent: Thursday, June 15, 2017 1:45 PM >>>> To:users@kafka.apache.org >>>> Subject: Re: Kafka Streams vs Spark Streaming : reduce by window >>>> >>>> Hi Paolo, >>>> >>>> That is indeed correct. We don’t believe in closing windows in Kafka Streams. >>>> You could reduce the number of downstream records by using record caches:http://docs.confluent.io/current/streams/developer-guide.html#record-caches-in-the-dsl >>>> >>>> . >>>> >>>> Alternatively you can just query the KTable whenever you want using the Interactive Query APIs (so when you query dictates what data you receive), see thishttps://www.confluent.io/blog/unifying-stream-processing-and-interactive-queries-in-apache-kafka/ >>>> >>>> >>>> >>>> Thanks >>>> Eno >>>>> On Jun 15, 2017, at 2:38 PM, Paolo Patierno wrote: >>>>> >>>>> Hi, >>>>> >>>>> >>>>> using the streams library I noticed a difference (or there is a lack of knowledge on my side)with Apache Spark. >>>>> >>>>> Imagine following scenario ... >>>>> >>>>> >>>>> I have a source topic where numeric values come in and I want to check the maximum value in the latest 5 seconds but ... putting the max value into a destination topic every 5 seconds. >>>>> >>>>> This is what happens with reduceByWindow method in Spark. >>>>> >>>>> I'm using reduce on a KStream here that process the max value taking into account previous values in the latest 5 seconds but the final value is put into the destination topic for each incoming value. >>>>> >>>>> >>>>> For example ... >>>>> >>>>> >>>>> An application sends numeric values every 1 second. >>>>> >>>>> With Spark ... the source gets values every 1 second, process max in a window of 5 seconds, puts the max into the destination every 5 seconds (so when the window ends). If the sequence is 21, 25, 22, 20, 26 the output will be just 26. >>>>> >>>>> With Kafka Streams ... the source gets values every 1 second, process max in a window of 5 seconds, puts the max into the destination every 1 seconds (so every time an incoming value arrives). Of course, if for example the sequence is 21, 25, 22, 20, 26 ... the output will be 21, 25, 25, 25, 26. >>>>> >>>>> >>>>> Is it possible with Kafka Streams ? Or it's something to do at application level ? >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Paolo >>>>> >>>>> >>>>> Paolo Patierno >>>>> Senior Software Engineer (IoT) @ Red Hat >>>>> Microsoft MVP on Windows Embedded & IoT >>>>> Microsoft Azure Advisor >>>>> >>>>> Twitter : @ppatierno >>>>> Linkedin : paolopatierno >>>>> >>>>> Blog : DevExperience >>>>> > > -- > Michal Borowiecki > Senior Software Engineer L4 > T: +44 208 742 1600 > > > +44 203 249 8448 > > > > E: michal.borowiecki@openbet.com > > W: www.openbet.com > > > OpenBet Ltd > > Chiswick Park Building 9 > > 566 Chiswick High Rd > > London > > W4 5XT > > UK > > > > > This message is confidential and intended only for the addressee. > If you have received this message in error, please immediately > notify the postmaster@openbet.com > and delete it from your system as well as any copies. The content > of e-mails as well as traffic data may be monitored by OpenBet for > employment and security purposes. To protect the environment > please do not print this e-mail unless necessary. OpenBet Ltd. > Registered Office: Chiswick Park Building 9, 566 Chiswick High > Road, London, W4 5XT, United Kingdom. A company registered in > England and Wales. Registered no. 3134634. VAT no. GB927523612 > > -- Signature Michal Borowiecki Senior Software Engineer L4 T: +44 208 742 1600 +44 203 249 8448 E: michal.borowiecki@openbet.com W: www.openbet.com OpenBet Ltd Chiswick Park Building 9 566 Chiswick High Rd London W4 5XT UK This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmaster@openbet.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by OpenBet for employment and security purposes. To protect the environment please do not print this e-mail unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building 9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company registered in England and Wales. Registered no. 3134634. VAT no. GB927523612 --------------7B9C862AD919005A59FAF245 Content-Type: multipart/related; boundary="------------30261EB0D60254F9C3F88F7C" --------------30261EB0D60254F9C3F88F7C Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

If confusion is the problem, then totally agree no point adding more knobs. Perhaps you're right that users don't really want processing-time semantics. Just think they want them until they start considering replay/catch-up scenarios. I guess people rarely think about those from the start (I sure didn't).

Cheers,

Michał


On 16/06/17 17:54, Jay Kreps wrote:
I think the question is when do you actually want processing time semantics? There are definitely times when its safe to assume the two are close enough that a little lossiness doesn't matter much but it is pretty hard to make assumptions about when the processing time is and has been hard for us to think of a use case where its actually desirable.

I think mostly what we've seen is confusion about the core concepts:
  • stream -- immutable events that occur
  • tables (including windows) -- current state of the world
If the root problem is confusion adding knobs never makes it better. If the root problem is we're missing important use cases that justify the additional knobs then i think it's good to try to really understand them. I think there could be use cases around systems that don't take updates, example would be email, twitter, and some metrics stores.

One solution that would be less complexity inducing than allowing new semantics, but might help with the use cases we need to collect, would be to add a new operator in the DSL. Something like .freezeAfter(30, TimeUnit.SECONDS) that collects all updates for a given window and both emits and enforces a single output after 30 seconds after the advancement of stream time and remembers that it is omitted, suppressing all further output (so the output is actually a KStream). This might or might not depend on wall clock time. Perhaps this is in fact what you are proposing?

-Jay



On Fri, Jun 16, 2017 at 2:38 AM, Michal Borowiecki <michal.borowiecki@openbet.com> wrote:

I wonder if it's a frequent enough use case that Kafka Streams should consider providing this out of the box - this was asked for multiple times, right?

Personally, I agree totally with the philosophy of "no final aggregation", as expressed by Eno's post, but IMO that is predicated totally on event-time semantics.

If users want processing-time semantics then, as the docs already point out, there is no such thing as a late-arriving record - every record just falls in the currently open window(s), hence the notion of final aggregation makes perfect sense, from the usability point of view.

The single abstraction of "stream time" proves leaky in some cases (e.g. for punctuate method - being addressed in KIP-138). Perhaps this is another case where processing-time semantics warrant explicit handling in the api - but of course, only if there's sufficient user demand for this.

What I could imagine is a new type of time window (ProcessingTimeWindow?), that if used in an aggregation, the underlying processor would force the WallclockTimestampExtractor (KAFKA-4144 enables that) and would use the system-time punctuation (KIP-138) to send the final aggregation value once the window has expired and could be configured to not send intermediate updates while the window was open.

Of course this is just a helper for the users, since they can implement it all themselves using the low-level API, as Matthias pointed out already. Just seems there's recurring interest in this.

Again, this only makes sense for processing time semantics. For event-time semantics I find the arguments for "no final aggregation" totally convincing.


Cheers,

Michał


On 16/06/17 00:08, Matthias J. Sax wrote:
Hi Paolo,

This SO question might help, too:
https://stackoverflow.com/questions/38935904/how-to-send-final-kafka-streams-aggregation-result-of-a-time-windowed-ktable

For Streams, the basic model is based on "change" and we report updates
to the "current" result immediately reducing latency to a minimum.

Last, if you say it's going to fall into the next window, you won't get
event time semantics but you fall back processing time semantics, that
cannot provide exact results....

If you really want to trade-off correctness version getting (late)
updates and want to use processing time semantics, you should configure
WallclockTimestampExtractor and implement a "update deduplication"
operator using table.toStream().transform(). You can attached a state to
your transformer and store all update there (ie, newer update overwrite
older updates). Punctuations allow you to emit "final" results for
windows for which "window end time" passed.


-Matthias

On 6/15/17 9:21 AM, Paolo Patierno wrote:

Hi Eno,


regarding closing window I think that it's up to the streaming application. I mean ...

If I want something like I described, I know that a value outside my 5 seconds window will be taken into account for the next processing (in the next 5 seconds). I don't think I'm losing a record, I am ware that this record will fall in the next "processing" window. Btw I'll take a look at your article ! Thanks !


Paolo


Paolo Patierno
Senior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoT
Microsoft Azure Advisor

Twitter : @ppatierno<http://twitter.com/ppatierno>
Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno>
Blog : DevExperience<http://paolopatierno.wordpress.com/>


________________________________
From: Eno Thereska <eno.thereska@gmail.com>
Sent: Thursday, June 15, 2017 3:57 PM
To: users@kafka.apache.org
Subject: Re: Kafka Streams vs Spark Streaming : reduce by window

Hi Paolo,

Yeah, so if you want fewer records, you should actually "not" disable cache. If you disable cache you'll get all the records as you described.

About closing windows: if you close a window and a late record arrives that should have been in that window, you basically lose the ability to process that record. In Kafka Streams we are robust to that, in that we handle late arriving records. There is a comparison here for example when we compare it to other methods that depend on watermarks or triggers: https://www.confluent.io/blog/watermarks-tables-event-time-dataflow-model/ <https://www.confluent.io/blog/watermarks-tables-event-time-dataflow-model/>

Eno



On 15 Jun 2017, at 14:57, Paolo Patierno <ppatierno@live.com> wrote:

Hi Emo,


thanks for the reply !

Regarding the cache I'm already using CACHE_MAX_BYTES_BUFFERING_CONFIG = 0 (so disabling cache).

Regarding the interactive query API (I'll take a look) it means that it's up to the application doing something like we have oob with Spark.

May I ask what do you mean with "We don’t believe in closing windows" ? Isn't it much more code that user has to write for having the same result ?

I'm exploring Kafka Streams and it's very powerful imho even because the usage is pretty simple but this scenario could have a lack against Spark.


Thanks,

Paolo.


Paolo Patierno
Senior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoT
Microsoft Azure Advisor

Twitter : @ppatierno<http://twitter.com/ppatierno>
Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno>
Blog : DevExperience<http://paolopatierno.wordpress.com/>


________________________________
From: Eno Thereska <eno.thereska@gmail.com>
Sent: Thursday, June 15, 2017 1:45 PM
To: users@kafka.apache.org
Subject: Re: Kafka Streams vs Spark Streaming : reduce by window

Hi Paolo,

That is indeed correct. We don’t believe in closing windows in Kafka Streams.
You could reduce the number of downstream records by using record caches: http://docs.confluent.io/current/streams/developer-guide.html#record-caches-in-the-dsl <http://docs.confluent.io/current/streams/developer-guide.html#record-caches-in-the-dsl>.

Alternatively you can just query the KTable whenever you want using the Interactive Query APIs (so when you query dictates what  data you receive), see this https://www.confluent.io/blog/unifying-stream-processing-and-interactive-queries-in-apache-kafka/ <https://www.confluent.io/blog/unifying-stream-processing-and-interactive-queries-in-apache-kafka/>

Thanks
Eno

On Jun 15, 2017, at 2:38 PM, Paolo Patierno <ppatierno@live.com> wrote:

Hi,


using the streams library I noticed a difference (or there is a lack of knowledge on my side)with Apache Spark.

Imagine following scenario ...


I have a source topic where numeric values come in and I want to check the maximum value in the latest 5 seconds but ... putting the max value into a destination topic every 5 seconds.

This is what happens with reduceByWindow method in Spark.

I'm using reduce on a KStream here that process the max value taking into account previous values in the latest 5 seconds but the final value is put into the destination topic for each incoming value.


For example ...


An application sends numeric values every 1 second.

With Spark ... the source gets values every 1 second, process max in a window of 5 seconds, puts the max into the destination every 5 seconds (so when the window ends). If the sequence is 21, 25, 22, 20, 26 the output will be just 26.

With Kafka Streams ... the source gets values every 1 second, process max in a window of 5 seconds, puts the max into the destination every 1 seconds (so every time an incoming value arrives). Of course, if for example the sequence is 21, 25, 22, 20, 26 ... the output will be 21, 25, 25, 25, 26.


Is it possible with Kafka Streams ? Or it's something to do at application level ?


Thanks,

Paolo


Paolo Patierno
Senior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoT
Microsoft Azure Advisor

Twitter : @ppatierno<http://twitter.com/ppatierno>
Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno>
Blog : DevExperience<http://paolopatierno.wordpress.com/>

--
Michal Borowiecki
Senior Software Engineer L4
T: +44 208 742 1600


+44 203 249 8448


 
E: michal.borowiecki@openbet.com
W: www.openbet.com
OpenBet Ltd

Chiswick Park Building 9

566 Chiswick High Rd

London

W4 5XT

UK

This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmaster@openbet.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by OpenBet for employment and security purposes. To protect the environment please do not print this e-mail unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building 9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company registered in England and Wales. Registered no. 3134634. VAT no. GB927523612


--
Signature
Michal Borowiecki
Senior Software Engineer L4
T: +44 208 742 1600


+44 203 249 8448


 
E: michal.borowiecki@openbet.com
W: www.openbet.com
OpenBet Ltd

Chiswick Park Building 9

566 Chiswick High Rd

London

W4 5XT

UK

This message is confidential and intended only for the addressee. If you have received this message in error, please immediately notify the postmaster@openbet.com and delete it from your system as well as any copies. The content of e-mails as well as traffic data may be monitored by OpenBet for employment and security purposes. To protect the environment please do not print this e-mail unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building 9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company registered in England and Wales. Registered no. 3134634. VAT no. GB927523612
--------------30261EB0D60254F9C3F88F7C Content-Type: image/gif; name="obet_logo_tiny.gif" Content-Transfer-Encoding: base64 Content-ID: Content-Disposition: inline; filename="obet_logo_tiny.gif" R0lGODdhSABQAPcAAAAAAI8GVpINW5QOY5UPa5UTa5UUXZYUZIwVa5sVcZgWa5gXZ5cYZEQa ToQaaosabZUacpcaXpoaa30bZ5Mbapwbc30caJkcZJ0ceaEcdUcdUaQdfI8eZnQfZa4hhWwi Y4gibJgieZ0icksjVXYjaYkjcpMjdJwjaqMjfVUkWmQkYXskapwkZV8lYFwmXk4nWWopZ1kq Xp8qa1MrXIwseFwtYmMtZXQtbKEtcIEua4Iuc6QvfLEvi1YwX5Ywe1YxYFwyY2IyZmUzaWsz a6MzcqgzhIs0fp00hnQ1b3w1dYQ1el45Z306cqo6jYI7e5M7gZ47h6Y7d2M8am08b3M8cow9 f6o9g2ZDbpZDiJ5Diq1Di2xEcXJEdKJEkqpEfHtFd4NFfKxFgrlGlWxLdHJLd3xLfJVLh61L mq5Lg3VMeZ5MiK5MjYRNfopNhbFThntUf4RUgptUjnRVfIxVh6dWkbNWjLVWlJxakbpam3pb gYRbhLZbk8Zbp45ci35hhYJih7ZikIxjjJRjkZ9jlKFjmLJkorxknbpmlYRni6tonYpqjrxr nLtspJBtksZtqqFulrdypY1zk7l0m8t0s511m8J1opJ2l7F2l613o5R5mcV5qpx7nbl7qMN7 pKJ8oZuDoK+Do7GDqseDq6OEpLyEsb2Ep8mFs9CGuseKraGLpqyLq7uMqKWNqcuNs7SOq6iS rM6StbGTsduTx7yUr8OUsMSVus+VvMCbs9Kbu8mctc2cwLCdtNSdwbeiusOkvcuku9WkwsWm t9ymys2rv7ysv8KswdKsw9WsydmsxtyszM2yxNKzxcS0xtO0zd20zc21y8a5ydG5x8u7ytS7 ytS70dq7zd270eG80s3CztnCzuPD1dHE0tTEzdrF0eXF2uHL19bM093M1eTM2+jM3dvO2dnT 193T2uLU3OrU4uXV4+TZ3t3b3OXc4erd5PDd5/Hd6eze6Oni5uXj5Ovl6fPl7ezr6+/r8PHs 7/Xs8e/w7/Dw7/Pz8vrz9/f2+P///wAAACH5BAkAAP8AIf8LSUNDUkdCRzEwMTL/AAAMSExp bm8CEAAAbW50clJHQiBYWVogB84AAgAJAAYAMQAAYWNzcE1TRlQAAAAASUVDIHNSR0IAAAAA AAAAAAAAAAAAAPbWAAEAAAAA0y1IUCAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAARY3BydAAAAVAAAAAzZGVzYwAAAYQAAABsd3RwdAAAAfAAAAAU YmtwdAAAAgQAAAAUclhZWgAAAhgAAAAUZ1hZWgAAAiwAAAAUYlhZWgAAAkAAAAAUZG1uZAAA AlQAAABwZG1kZAAAAsQAAACIdnVlZAAAA0wAAACGdmll/3cAAAPUAAAAJGx1bWkAAAP4AAAA FG1lYXMAAAQMAAAAJHRlY2gAAAQwAAAADHJUUkMAAAQ8AAAIDGdUUkMAAAQ8AAAIDGJUUkMA AAQ8AAAIDHRleHQAAAAAQ29weXJpZ2h0IChjKSAxOTk4IEhld2xldHQtUGFja2FyZCBDb21w YW55AABkZXNjAAAAAAAAABJzUkdCIElFQzYxOTY2LTIuMQAAAAAAAAAAAAAAEnNSR0IgSUVD NjE5NjYtMi4xAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAABYWVogAAAAAAAA81EAAf8AAAABFsxYWVogAAAAAAAAAAAAAAAAAAAAAFhZWiAAAAAA AABvogAAOPUAAAOQWFlaIAAAAAAAAGKZAAC3hQAAGNpYWVogAAAAAAAAJKAAAA+EAAC2z2Rl c2MAAAAAAAAAFklFQyBodHRwOi8vd3d3LmllYy5jaAAAAAAAAAAAAAAAFklFQyBodHRwOi8v d3d3LmllYy5jaAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AABkZXNjAAAAAAAAAC5JRUMgNjE5NjYtMi4xIERlZmF1bHQgUkdCIGNvbG91ciBzcGFjZSAt IHNSR0L/AAAAAAAAAAAAAAAuSUVDIDYxOTY2LTIuMSBEZWZhdWx0IFJHQiBjb2xvdXIgc3Bh Y2UgLSBzUkdCAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGRlc2MAAAAAAAAALFJlZmVyZW5jZSBW aWV3aW5nIENvbmRpdGlvbiBpbiBJRUM2MTk2Ni0yLjEAAAAAAAAAAAAAACxSZWZlcmVuY2Ug Vmlld2luZyBDb25kaXRpb24gaW4gSUVDNjE5NjYtMi4xAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAB2aWV3AAAAAAATpP4AFF8uABDPFAAD7cwABBMLAANcngAAAAFYWVog/wAAAAAATAlW AFAAAABXH+dtZWFzAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAACjwAAAAJzaWcgAAAAAENS VCBjdXJ2AAAAAAAABAAAAAAFAAoADwAUABkAHgAjACgALQAyADcAOwBAAEUASgBPAFQAWQBe AGMAaABtAHIAdwB8AIEAhgCLAJAAlQCaAJ8ApACpAK4AsgC3ALwAwQDGAMsA0ADVANsA4ADl AOsA8AD2APsBAQEHAQ0BEwEZAR8BJQErATIBOAE+AUUBTAFSAVkBYAFnAW4BdQF8AYMBiwGS AZoBoQGpAbEBuQHBAckB0QHZAeEB6QHyAfoCAwIMAv8UAh0CJgIvAjgCQQJLAlQCXQJnAnEC egKEAo4CmAKiAqwCtgLBAssC1QLgAusC9QMAAwsDFgMhAy0DOANDA08DWgNmA3IDfgOKA5YD ogOuA7oDxwPTA+AD7AP5BAYEEwQgBC0EOwRIBFUEYwRxBH4EjASaBKgEtgTEBNME4QTwBP4F DQUcBSsFOgVJBVgFZwV3BYYFlgWmBbUFxQXVBeUF9gYGBhYGJwY3BkgGWQZqBnsGjAadBq8G wAbRBuMG9QcHBxkHKwc9B08HYQd0B4YHmQesB78H0gflB/gICwgfCDIIRghaCG4IggiWCKoI vgjSCOcI+wkQCSUJOglPCWT/CXkJjwmkCboJzwnlCfsKEQonCj0KVApqCoEKmAquCsUK3Arz CwsLIgs5C1ELaQuAC5gLsAvIC+EL+QwSDCoMQwxcDHUMjgynDMAM2QzzDQ0NJg1ADVoNdA2O DakNww3eDfgOEw4uDkkOZA5/DpsOtg7SDu4PCQ8lD0EPXg96D5YPsw/PD+wQCRAmEEMQYRB+ EJsQuRDXEPURExExEU8RbRGMEaoRyRHoEgcSJhJFEmQShBKjEsMS4xMDEyMTQxNjE4MTpBPF E+UUBhQnFEkUahSLFK0UzhTwFRIVNBVWFXgVmxW9FeAWAxYmFkkWbBaPFrIW1hb6Fx0XQRdl F4kX/64X0hf3GBsYQBhlGIoYrxjVGPoZIBlFGWsZkRm3Gd0aBBoqGlEadxqeGsUa7BsUGzsb YxuKG7Ib2hwCHCocUhx7HKMczBz1HR4dRx1wHZkdwx3sHhYeQB5qHpQevh7pHxMfPh9pH5Qf vx/qIBUgQSBsIJggxCDwIRwhSCF1IaEhziH7IiciVSKCIq8i3SMKIzgjZiOUI8Ij8CQfJE0k fCSrJNolCSU4JWgllyXHJfcmJyZXJocmtyboJxgnSSd6J6sn3CgNKD8ocSiiKNQpBik4KWsp nSnQKgIqNSpoKpsqzysCKzYraSudK9EsBSw5LG4soizXLQwtQS12Last4f8uFi5MLoIuty7u LyQvWi+RL8cv/jA1MGwwpDDbMRIxSjGCMbox8jIqMmMymzLUMw0zRjN/M7gz8TQrNGU0njTY NRM1TTWHNcI1/TY3NnI2rjbpNyQ3YDecN9c4FDhQOIw4yDkFOUI5fzm8Ofk6Njp0OrI67zst O2s7qjvoPCc8ZTykPOM9Ij1hPaE94D4gPmA+oD7gPyE/YT+iP+JAI0BkQKZA50EpQWpBrEHu QjBCckK1QvdDOkN9Q8BEA0RHRIpEzkUSRVVFmkXeRiJGZ0arRvBHNUd7R8BIBUhLSJFI10kd SWNJqUnwSjdKfUrESwxLU0uaS+JMKkxyTLpNAk3/Sk2TTdxOJU5uTrdPAE9JT5NP3VAnUHFQ u1EGUVBRm1HmUjFSfFLHUxNTX1OqU/ZUQlSPVNtVKFV1VcJWD1ZcVqlW91dEV5JX4FgvWH1Y y1kaWWlZuFoHWlZaplr1W0VblVvlXDVchlzWXSddeF3JXhpebF69Xw9fYV+zYAVgV2CqYPxh T2GiYfViSWKcYvBjQ2OXY+tkQGSUZOllPWWSZedmPWaSZuhnPWeTZ+loP2iWaOxpQ2maafFq SGqfavdrT2una/9sV2yvbQhtYG25bhJua27Ebx5veG/RcCtwhnDgcTpxlXHwcktypnMBc11z uHQUdHB0zHUodYV14XY+/3abdvh3VnezeBF4bnjMeSp5iXnnekZ6pXsEe2N7wnwhfIF84X1B faF+AX5ifsJ/I3+Ef+WAR4CogQqBa4HNgjCCkoL0g1eDuoQdhICE44VHhauGDoZyhteHO4ef iASIaYjOiTOJmYn+imSKyoswi5aL/IxjjMqNMY2Yjf+OZo7OjzaPnpAGkG6Q1pE/kaiSEZJ6 kuOTTZO2lCCUipT0lV+VyZY0lp+XCpd1l+CYTJi4mSSZkJn8mmia1ZtCm6+cHJyJnPedZJ3S nkCerp8dn4uf+qBpoNihR6G2oiailqMGo3aj5qRWpMelOKWpphqmi6b9p26n4KhSqMSpN6mp qv8cqo+rAqt1q+msXKzQrUStuK4trqGvFq+LsACwdbDqsWCx1rJLssKzOLOutCW0nLUTtYq2 AbZ5tvC3aLfguFm40blKucK6O7q1uy67p7whvJu9Fb2Pvgq+hL7/v3q/9cBwwOzBZ8Hjwl/C 28NYw9TEUcTOxUvFyMZGxsPHQce/yD3IvMk6ybnKOMq3yzbLtsw1zLXNNc21zjbOts83z7jQ OdC60TzRvtI/0sHTRNPG1EnUy9VO1dHWVdbY11zX4Nhk2OjZbNnx2nba+9uA3AXcit0Q3Zbe HN6i3ynfr+A24L3hROHM4lPi2+Nj4+vkc+T85YTmDeaW5x/nqegy6LxU6Ubp0Opb6uXrcOv7 7IbtEe2c7ijutO9A78zwWPDl8XLx//KM8xnzp/Q09ML1UPXe9m32+/eK+Bn4qPk4+cf6V/rn +3f8B/yY/Sn9uv5L/tz/bf//ACwAAAAASABQAAAI/wD9CRxIsKDBgwgTKlzIsGHBd9qAtRJV qaKmTqhuKQO3Tl44efLKgSsnb51JefX2OVypkJ+2VofcuKmz59CiTqJawYJFkZGmZJxotZt3 T9y+e/juzZtnzmQ9llD90cNVJ4pVq168hEGDhqZNTbZ42TplSE0rdvnuhWPXbt+5dv7u7TNH MipDerC8sGAhAwcRL10BHZJUSZOoTpUWLTK0yJSsU44YLasXj9u5eJf3xfPHr905cE/tHkSG JoIBFkSidD1UqZNr1zlh4QImTKImR440abIDKZekUuz2saO8b9+8d/TCrRNN8F0lGQYu4FAN aNEhQHW4htlax2ulVsCuXf8DZsqRIUN4+Ni69IsdyHnFheOj1w0cc3/a3ESIgNqLYDdR4HDC BQRewIAEIuCAgxV2nKcJLNeIk4wsk+CRniPGTNPNNPHEV89a4ohjn13IRGFABDJEMVMULDBg wAEwHsDAjAxccIACEkjAgxh4LAIJLN6gI8wpeGghhindbHPLLNyYo44887xFTzXLQYUMEdHh 4AWAFxhA45c0LpCjBDjmWIEVhhSWzDveEFlEEZwMM4syxZzTzTn1FDXPONGE5tCVp2npRZdg FipmBhIUoEAFZGawwQ526GbLO+/wwsgORZRyyyrB3cNOOKCWAggw8qykjYkXyOBFFPsVCuYC h47/mWMGGaCwA6aamGILOu/owkgIR+jSTTYpFXfPL6tUIwmeDbXjxomqYunqqwpUK0EFCtCK QgUVbFDEDgmgwEgrtogDjy6FPJDFLeF0w1Y88fzCiW/tVNmSJPux4IW004ZZLY7c1kprtxs4 WgEEEDAiljjp1JIICHFwuAwtw3DTzjS0aCMiQ8BAx0IU/PbLwAL/WpttwRts4IEHG6CAAcIm cMKLLu1408oTK4Syzy/LrNKhW/6IY85C7ngRXWoRiDwyyf8WoGgC2Faw8gYYYIACDyhAYIIP tgBzDDzOcLJCFcZkQ4sxwwkXDz7mqJQQKhfwF0XcIsNaLYwKiIDpDigk/5DA1N2GgEIIWqt7 zDHdpKMLITQwUs1w6gzHjjhsu31QO0ZfQAR0SU8LqwQw4oBHK8kcg8tYhjThQQJVvxxCCCCA MIEFkDTTTDrdkJLFE7TcyQ2ot5RSTdsJtRL3xwzs1/mrOR4ggSbo8CNQPeqEc444wEzSBMIV PFBCCCaA4IAFRwDTDDXm+JKIEZioc08954SzSi5oW17QPWgcfYLyy4eZ4wBQ8EZB6iGPeNRj GxoCxhoQ8AAIlOCBs6PBERhBjWaQYxqhGMIXlFGceHRjGLDAh70M4owTnIYIcduPgb4kJgkM wA74MEg91vEpdpgjHPVoBh0c8AAfQIEGIKCBDv9KYARfnI8cvZiDDUDBDnZESR3t4Ec5EtKJ pMlABgzYiwxWOKMWDmAP0jOIPLjRxH1Ejh3nsMbu1KAGOuBsBSsgASam8YxtFIMSMBhEN8IR uWp0ghYjJAg+8qe5C5wgNURgoQJOcAArxPAgY8TTPd4yyXP8wo0lYGMJSECCFYChGc+QBjNi QQUuSCMe2eiGKIARDD8VZBw4OCEDVIMGHMhoaRK4gAQEiBB5gIMt8EvJh8LBCR/QwAxmMIIO dHCDJPgilNi4oxQwcYtpLAMVpeBGQpARtxNscSZ1YOQFmHaCARhCIeyQxj3uEQ8OxQON89Ah He5QBR0kgZk6cAU1qCH/jV6o4gukmAYoviGOctivIKKQkQy8eYhD1IEBB2CaBBjJS4TE44by UMYqhnEZY9BiGnfIQhagkAQjPCELVQhoKIsRiylAYhmSQAdIFLIIL+HgAlFoTR0iWq0CiEAE VghjL6EUjjndIh7hoEU2hsGJJpzhCFXIghK6YIREQIMZzOjFKKbQB+Fxo1QKqUN0TsAANIhC FGu4UU9vxYiFRDIeyrjFOfZRj2XwbBBGKMQTkOCEkUJhDlhlxi5UMYQ0mIMbrjwIP9BwAEMy YA86CYNaFVCAOmihFW5dx2XMMddJ1iNKj6hCIZTwASQ8gQo3uANWi7GLV3CBDNuYIVgRwg/J /x7oAIfABS4k2zRDrIEXblXHObLBDnJghh2YOYcvlBAHJMBgCFXoQhfuQIxiEOMVrJjCFshx D3jQ5aADqa2MGGkIYABDC5MtgG9t4VZuiGMayF3LPKx3DmkMoQ9CQEISsIDSRFi3F69QBRjA oIxzvOUc60isQBYrI9AZAhnI2GnT9mAHU7gVHNabxjDQOI1wILUYN1AiFZyAhTaAgRFX7QUr YtGGNpjjncHxx0wNstMDLWAPznCGIdK7BkacUyH14MY0yLEMjnKDG1FqxyioUIYUDMEGQ8AC FjzBDNZ+IhVbKINl0saPd3ADvDsmkATC4AxraEJRa9WEFoR6kHqAw//A3eiGMoJzDnbc4w1l SEMMhAAEKAvhE70AcCo+AQQy7LFD9WDHfALpD1EMYAG63IEzrsELMv2rAmpOhkLmwQ3jqqPD eGKHOsBhAz0EQQhBKIMcpiAFVuxiF58YhSeMQIjgwE8cKYGHewoCjIgaUgK8EM8O0EzZHtNB IR6pXjjmYUBQ3cMSU4BDDWbwhjfI4Q1peMUrUpGJVEDiBoOwXpwNHI95oGNoBEGHBGB1ggJo whveMAQBqpWAAsQsC8BISJBXcVThCOcczBhCIKgwbThsQQp6QAQrRvGJSMSCEEwYhDiayGw0 isMZ4LAfP3YQ0RxZAd68SIACEgABp2WBE1D/QAdC6uGKuL44fuowRxnK0AclCMEGXNCDHriQ iU9sIhKZuKMZhrFH5N5DH/UYBi2GUxBGPPoCIlAAL+CtBQKUvAAMTMQisgAXMcZVGXIJph6k 0IioGiEIQJgBENIQiUhYAhGjyMQbKDENpagjHu2IhzrMXa+CeINMUC/AHkLEC6cV4AGIBwEm EoGFbojRHJ+CTz2k8QYpKEIIWChDHMBggz0rQhGICD0rxrCFW/yiLYnGzN3bcQ91GMQOBdAl t4ARIjswEPEOcAAIEoGJKoRiHgRJNPXip4opvEERP6iBGuYQhzkMQQhpQEQe8vAGT/hCD4Hw RS5ogesPnQM++Cio/0GSUa3AryFEzvABA3PvgAms4A6hoCclntEWdqzDHMzYxMwbIYgg1GAI ZWBibzADQvAGcJAGZJAHo8AJc7AJv/ALt5BK2dAO7RAO30AP7TAiBMEPa1AAOSICBWAKIVIL D8B+FmABE9ABSfAImDAIbcAGMAgHZUAGgQAKozAHSeAEVHBqZTAHXyAEW5AGW8AFY0AJsYAF SEAM3fAL8DUsuHYU9CAO0nAQySBy16I1v1BBmJB7J5iCHdABpTUHj0AJZEiGlkAJaQAEbAAG TjAEM6B2Q3BqUiAEQiAFcsAKgqACgYBcybUU1ZMNBCUJ2mQQ/IAHBEABFGACCOADteALvf8w CCjohWD4AR+gAipgAzZwavkFBjPwAl8ABkkQBGgHBHz2hmo3Bp/gCdNWTcMQDuZQLPrADqmk DbSgga/kAwWAiCbgAE/gCqqgCn2wAl84iZVoiS5wjP/nBGzABVTABlNQAzZQA0EwBUIwbWp3 BZbwCnBgA6MgULfgCmh0DubAWZ81cQo2ELZAAQhAAbEzAVVACZtACX2QA8RoiSpwjC4QA503 BFMABl8gBUMwbXQYBJn4hlfgB6+gCFOQCe3gC3IFCnbyFsUxD1FEPLTFCDzEAe2oA30QCIGg B1RAifaIj6b4AgQoBDNQAzUABCoZBCapkleQB6ngBzWgB+zgD+z/4AqPMAvhkA2rZ2BJwWgF QQ90MAG6BwLCSAJfUG1pwAVDYIz5aIpvqJLS6H8rqXadCARb8AesgAhDwAYZ91kf1IqgYmDB gQ/MshDtoAYTMAGxI4wdwI8HJwU2QJJSaY02QIduiJUqOQaNkAp5UAN3MAvnoA7dIA4eFmfk AEXhQA7ocA9CeRDoEAcd4JY5sAKU+AE2AAQseYxSSYAqOQNDEIrR+IYxsHaWoAp64AJlQAur kA0GJFx5NyxMOA/8sA/lcI4IAQ+D8IU58JvF2AIxMAPD+Zn+BwSn9nzVuJkIJ2tpUIDSkA2X sVnq8A2eYVc45A8EBBX4EAo38AEd8Js3/1CXnvmZKWmV0ggEcchVmzAKgSCKhABfddZEdpJo dBVn7cA2swUVysAGlngDOcAESNB55mme09gGgjAKqtAIUjACZAAKEYgZxQI/opZ6h1kS9wEP qkAFxxgETOCPXMBqo4icQSAFQRgIlKAKnhAIUhADVGAJ0nAOy6AMoAIvwlcP5MBd+DAP2LCf zEEOo7AFM5ACNTAFX8AGcDAHOqcHfxAIjVCGf7AFaScFkcAMe9QOiNlh8FJDGFMNF7UO4HUf 7LALfiAFnXie2sUFZcAFWzAF/qd2MokNlGEOG2I9w8AhFDcPqNANxZBg96EQ+4BEmZAHV7AE P/CGpCgFV0AGiv+QCswADvqgGa9YD+agDMOwCqAQDfEADubgEbr5pwixD/JgDttwVdKwDeDA EShBQCDRqq7qHihRHKA6q7Raq7Z6q7iaq7q6q7Q6D+CwDTfJq5zBD2zWEOKgDY9UEOwQCUAw AiMwA3IwhbpKDyCTGnWADA1xDycgAM5gEOQgBRowAoo6AxrwArswrRLgJQcQAAKACgyBD9t6 DQVhD1egAVcgDfgwF38wAi+ADf4wD+QAfNtADNtgEPiADcyAbgJRXJ/FDNBgDwJBD9sKDOiw CAJwAsnKD9eADBU1JSYECyo3EK+gAUsQrAuWBxqQB/6QChowBijbACMgBya7C+DaAC//gAjA 5w9yMAJjsAThKgX2IbECIEDjcLH0IBDAQAQCwK5hoHKdEAAwIgBeQBAomwoGsQ0v0AP18ArO OgN+gAg90LIqwbIzoAifMAYaIAcqkQfOegVyMAMxKxXbigvXUAcC0AkCgQsCYAB1UAleIABE MBWJdAA4sAgD0Q9boAHMIEPNag5cOwPQIBDb8AMaQAzz8IYFOw+RMAIN0Av+wLZjEEOfMAJA MA8SewACkLonIA7+gA/Qga340AkGEAB4K7TdShBoSwwGMQ8/8ALs8AoNgAgE0QgN8AnEELzs YAnkOga7EEMomwkCIQ0j8APyILEGsAZ7gCrXoA0CEAbt0Akn/xAAOIAKR6ut3FoQjaABwlsQ vTACS7APrNAAkUAQn9AAltALJEuuckAM+4APT4GyliAQ0DC91but2sAZixAAYYAMF3sBAUAE toAP8xEX23q7AyG95koQ5OCz0MuyW+A2+4C2r0AOLxCz0koM0Pq5GhDA/jDA1Cu0B+wP1yAA OKANXRIF2CrDmsMP8Hq+BWEJ5YoIxMAMhGavwJcKzvoH27ANiPCsQyMHKQt5u0Cu5wrAAkzA QmsM73ANaCC1/rAHARAF2tAOuBC+eGu+Fny4lsC5GqABDaABaRCsSPwDzjoC4Xqu/mAOUgCz MwCznyAQciC/AsEMGjADBewlF7C0B3HQre8QBey6rgJwCNIjsQGQxgSBDZYgB3KACL1gOanQ AH/ADH4QfdIqEPJwtmMQCQUrEJmwBa8guWMgB/OADwGiIFGwB/IqEPcgCquCYwOBD2sgxlHx yetLq8RKrMIqEJ/sB8m8q72wBX/czNI8zQsREAA7 --------------30261EB0D60254F9C3F88F7C Content-Type: image/gif; name="halfcircle_tiny.gif" Content-Transfer-Encoding: base64 Content-ID: Content-Disposition: inline; filename="halfcircle_tiny.gif" R0lGODlhBAAHAMQAANijvtCSsvju85gLVOfH2J4eX40AQ5QDTaMmZ5AARp8kYJ0bW6IiZZYH UZUETtScubdciLdWh6IlZuvR3pwbW8qGp/fr8dimwP///wAAAAAAAAAAAAAAAAAAAAAAAAAA ACH5BAAAAAAALAAAAAAEAAcAAAUX4DNhWKEAGJNQhDQcAdI412JEAlRZWAgAOw== --------------30261EB0D60254F9C3F88F7C Content-Type: image/gif; name="sig_promo_img.gif" Content-Transfer-Encoding: base64 Content-ID: Content-Disposition: inline; filename="sig_promo_img.gif" R0lGODlhwgFLAPcAAF0nXlIqXFopXlgwX2AmX3keZ1stYlYxYFs0Y146Z2QkYWwjZGMsZWos Z3kmaWIyZmYyaWwzbWs7bWU9a3MzcXM7c3w2dHI5bWVCbmxKdGpEcXxDeHRMeHtMfHdGdnRS e3tUf5gAXZgaXoMcaosbbYseZ5cVYJgXZ5gZY5gWa5oba5QZa5wbcpcZc48ccKEddaUdfIYj a5wjZZ4iap8paZ4oZogmc5kmd4g4epY0fYExbKEtbaIybqItc6Uhe6QoeqQ6d6Y3eIZIfYVT fqtDe61Jf3tdgn5VgX1gha0fg5o6hpQ0gqkpgqo8hbUnjYpJhJdGiIlUhoZZhpZXjJ1WkqlO g6xDiadJirJJi7FShrVajbJTiqNHk6NXk7dal41kjYVni5FljoxukZdpk4xzk5x7nZh1mK9g jqVmmbhhkrtkmr1rm7hmmKJznKx4nrRwmbh4nrproqp7o7l+o7R4pbZRocFonsFbosNsp8J0 osR7pMZ+q8R1qtF/t5qCn52Gop6JpKOEpKOKpqmKqb6MrbeGqaqVraeTrLqUrrSSr7mNsa2a sbaYs7WkubqrvrmjuceFq8KTrsyLtMmEscOZtsmVt9Kau9KTu9KNusOitsmlvs2rv8amutCm vb2vwb+xw9ecxdKTws6swcaow9WkwtmryMSyxs6zxM2xyMm8zMm1x9G0xdS2zdS8y9q0ytK8 0d240+C6z+K808zCztXCzdfEz9XF0trF0tzL1dfH2t7O2NXM1trS1t3U297Z3OPM2uPF2OLU 3ePZ3+jR3tzT4uvL4OrU4uXc4und5ODc6urX5fDe6OXj5Onj5uvl6ezq7Ofi6PLj7PDu7+Xm 8uvs9fPt8fju9O/w9/Py8/nx9vT0+fr2+f///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAAAAAAALAAAAADCAUsAAAj+AKcJHEiwoMGD CBMqXMiwocOHECNKnEixosWLGDNqnMito8ePIEOKHEmypMmTHTeqXMmypcuXMGPKnClRG8qb OHPq/Eizp8+fQIMKHdpz286jSJMSXcq0qdOnUFXaTEq1qsmoWLNq3cq1p9WvYD12HUu2rNmz A6mFXUsVrdu3cOPSZEt3p9y7ePPqbTi1rt+rewMLHvz2r8dsRg2LJcy4seOndKN1gqSFSJAg lrOk0ZOJ1iZftmwx47VrNLNsSh+rXs06ptqv21xpAcKDCBEgQIJg8WJHkqVLeNZEQubslzFs uLA948XL2dHW0KNLr/jVVREZMmoXKZIlixY7dtb+2MmDp88lPnFa3RIlTBg2ZNva+0qMc7r9 +/gL9j0ajc0OEbVtkYYWWnSXhRffYUJKKZJMggceXuCRRybNQNOMcsdE44tz9eXn4Yes7adT LESIsAMPRVTGww4zoICCCj300IQXd/BxiSygYHLHjpSssgo00mAjzC2u4PJMhyAmqaRgSHXC A4BEZPGfCC66eIKLKqjwAhNYjAdKMaDcMaMoiSCySi3OHPPLMLbQB9iScMYJl5s3OWliEUBQ iQKVV1b5ggoo/AnDD17sgUkxpeAxIyGIYNOMLrrUAkkmvtwk56WYkiXiSbHwIMMORMggwqh7 VnnCqT5oqeUNTCjBhKH+xcBCBxNqrHJLkNVQ0skbwVia6a/AQpVTNCXWUOKoeprQ56knqDCD DzDA8EISMCTBxByhAFNKHDkU8ouPt2xSCCS4MINSsOimK1RObIhgLLIu6okCsyekkAILSbwA gxPWtsCCEpKQYswoXeCwySaZrILNL9LUUulJ6kYscUybjuTKDtmJSioKytJr770vpLBvtEw4 wcS/pLBSzCU4oNGKJrg08wwu0dRy7sQ455xRxSJlIUJ28FY5L7M9WOGFGnU0UW21LTDRAgkj 1FFKLrJCQccrt+BSSyS3CHOzzmCH/dBrJl0M9KgmCI2CvU1YMgw3zgjTCimSeNHC3S4s4QL+ CUuAwkoulHCBBjLS4NJJJrH0CrHYjDeOEErt1jClnlc2e8Ia1niUpjOprOIKHS60cMMSJNig RB2v5MJKGxZoUk0wrWySjeJvOm574ydFk+fkVs6rwgmSgMQMLr0sJwwscSwBBRdK2FDAEqOg kosiFLTBSjDBWLLJkYvf7n3YJ5UiAwo8oEDDDstmuUZIzdByzDO6wN+KF0soj4MDFtCRyiue RPFFMqvYxKR89b0C4oxnHoHEz2pQGyAMrVk02FQzbNGMIb0HF4SoAhSgkAMcUIAKrHjFJ+QQ BTlEohOb4AUBDcjCdCGwI1r4mQyqMBtmAWoSIhlNL1rBtWNoohP+ULgCF3CghNOFUBWMeIIi 4NAMc62whVDMFNlG4kBRiacHp0rBDFRQjBwegxeZQEQwdIGITEChDkqAQhGVYIpUeGIQFWjD Lmj3tSjaUU5TFEkPrKQHSLTIXjnowUiG14xbBEMawkAGL7rQBSVQYAlE/IQpGJEICbiBF058 okKSgYpOetKTydBIFSixkFEOBBWECIop5XKGVlpkFK3UxFZamUqEoHI6JcnGHq8kCUn8jm1N GGQwahGMrh1DbmigAgUagAMucMETpjiEIB4QiETyohmaTAgqQsDNbnYTFRlJRghqiRBxkpMQ IQjlSyhBSoKY8y7dtAgluNnOrHDzDAn+QScxpGOSHaRNBZawxAw+ZoVgimSC6rkFL4qJi0FE IQIWgMIToIAKVRwiEUKQA/aE8QwO1W4h26TEJ0EZznFu0qQCIQY4YVKFKhTknax0ZUXmGYJ6 YuWeCVEpLkuyAyt1ohQ9+FgTrJC5kDyDFp7ZxC1OcYxXjGEKDLAABbhABVOYAhCA2MAggiGM amwDF9LoHkO2udKWEAOl5UTrTFpakLOSUyW5SN1CiPGKfabUFnY9SDLqepFXvEKdAqEpKema 14LIVSFxVQhdAVuQuBZWIDhdkkmskIIXgaIUTfjYD+JQCpE44xbIoMWZnnGLZ1QgChCIwAMk IIVGeEIQgYD+ACN0gYxnZCiTJHkIWRVCiSokI4MzqIIsCaKJN7T0DDadxjh/W4UZnGG4AoHp NHo7kFdU4RWjaG4V3ioQVJwhuG+gq0sRUgUWtXS8yiVELs4AhCrA4bHEMK5zy5oQSgChm0Cw KU0J8YZuUiIZZ+jmG6p7T5qG4AyADXAIGJKMOXjzDLkILDcV0V9uvpUQ9+3mHBJMzypwcwaj GMh+K6zW6c6gmxAmSGQPQl2CxFe7yZ3DHAoi44HU2LuFCKxvgVuFEBekt8ElZy6qQN9pWLfI ETEJG1JgghtMAnkpaEGU+eCFHNJimF3r2ig04AEIfEAKRxDDIhZRBjlAQRG6wEX+K475C6+V RLchQDJB0DmHMxAigyGA7lntTIgAzzilIZgDDe/s54FIF53q3OYcgHDn/v5Zwnx+wwzmydg5 54DRdx5ICN4AhDkQwsHonUYugkuJUfQ3uTT2poZFrGr8qrqdr2i1hQei4IUkw8OqBoKEZV1W EqOY1rKegV0NrOqyEvvDKj5wPhc8kFFjur9VyCtbCTLtabR0np0WCJ357GHoTsPBc0CFfaMt kCrgkyCcvgidQCKJys7ACziSspS50AVZsC8VhPDFM5oxzA1MIQIBkMIDpCCIQwBCEGh4Qiba cwxdBKMTu/ioQkI6UlTkFZ0DHkgVdG1ogswzwpBVQY7+bRzn6KIV0d0NARAA62CQK5ogrzhx pakd6mmcNQSvGMgoaqpx3w6EEMK25YdH8QpNnLjk0/0wJYzeTUKgAtdaEEisuVkIVFSY49Oo tc39ynW7OjgERGaFpEFO0x2UGtePRiUqin50kCsYCJqgxNGHS9NJawLtgAZ7LohB00ev2CD6 TOkMMi71GdCh5wWpdksnPedNUxvrmuC54Ec+z5wPZAbclYhJgFEvFbBAFrLI7N1aEIcrREN4 ZSzXLXphhig8AQcQ8MAHOPAHQJDBE2aggi5ucYxjXEMXn5G4NmV9cpwTRBHMdmcoxdlOcWJ9 IEA496HTmXKb7vbbMzCIg2f+jngXK9vQKL0+oFEtEAVb3sjczPh+Uw72vKNB6twc782Tr/Wp +xeyIdgBYPNK00iw/9yGtk87J3kKtlLoJHnrNw3b9H00VVYeln34B4CAR31JB3I/FwIgV23l hl4exggFgXznNw0oZ20SiH3QR3gfdxEnEQSV5WTAgAl3kwIrcAN00AXG8BHMcAu04AzSsAtl wAEgsEFCYAAGIAZmcIQ/WAjCgEjNgAvBgAzBgA1v5hBklQxWeIWNx1jbdHG4Fn0oJU6KYBDp Ng3Tl2hIZ3IrpYEpx30baBAltl3aNk53NofRhxDc9HzKFQIQKFjwh1aRNXUAiGvAxmy28Gro F2j+CNF/Jvd9JuZqBMhNlreA7cSHZBhZCtZKrXR0dvV3BTGCaoh+aVhzipd8AxEJFLiGlchn c9hfljdPdnUGUaduJnEJKXACPvADsgAMTbA3UGMDdHBmReUMcUMLjDAEU9ABBkAFUCAFEcAB YgAGYFAIbgAGlMA18BMMz1ANt4BbIQFncnaBWhgCdlUIZpcLnDRPtSRO/lcQhQCBZZhy9CVO KzWGHneKBqGGbpV4tWSK59WPj1YQdwiQerhrsEZPmvZ9gDiICmlz7NSQOTd1h8diBhmBu8Zo izaR5teHkziReXhuCiZrIMeJWdiGjYV0aqh4NSeC9rhNy6dy/Xhelpf+DJgnaiHgYxaROyyI AizAB4hyAyQANSMQA2PwBk8wBoVgCIPQBkIwBmbwBBFAAR7wBCAwcGD2BYEwBWYQd8GwCr+A DG0GDNwIEt6oECOYcuPnTii1Z/c4Xu+ogGdYiSs1TzZJhvfFhtaWkm9YSynoEBnZh+o3kVNX T3+4YlqXddy0ENykfwPBfxxpidykToFZfpCokQR5kOfWgAkhkuAoEO1oEDtneSfJgSlZlm6p TkBAeAgBdCKIhxSBEpMggzPQAqQADJIAlAVwmxQwBW1whLvZBlHQAUKAA0SoARTAARwge2Uw BhRAC8JQC11TC7RwC78ACWHJE1T4lhMYji3+aX0hEIaV+JYhZXLnRIHiB5caN2nEkAzZ1V92 qQU1B4b6GF078I+iNnx6qAlsN5lJJ3mRSZGHCICFWZgIUQjxt3YtV5n+qWDqVGHt1Jf9SYkd mXczAE658Ab/qJlxaIbcVW1nIAN2RQ2FKJrZSRAsqW1BZ2iMlQs1BQTeOQ3UQAzUwBEnYQ1N 0AIncAM3MAqjIAcjcJsOUAALsAAEMKQKQAFCIAASsAERgAAQAAESgAABgAFiIAcJYAackAm1 8Az7pgut4AqZIHxCJ1IVZ1ekWaLWBgSoYIX2FQLrKE7spQi5ADjBBX7jaYbxeIYN1k2TZqYS KaYm16LlNnLTwAj+gWaOr9BfFqh9skZOlNifEfqfC2mYpKhXuOZNHAehkVV333V/kmp5D9qY jHhsoFqCIykQ4FZX2TUDFqgKIaAFFscIuCGinbiSFHhrcJcMfDd4BVFhFoh837gQN1EKN3AC K2ADNjAISfmjQLoACjCkBOAAFLABQtABFXAAqRUBBnAAEmAEhhAFUrAKiLAJtOCch7RvD5Nb 10l8tVSmtfqRZxBr6cimuZBhgZZXbVme8thWnZShCQFgkXVW66hxgDoK9PquC7Gmd0hfjTqq fQighymZk3oQxPB13LRhCPqoKllgGKmfnyp5GKuA9JpfyUaqmyliR3dgidqIKpcLVyD+q6WK itFFYjNZkqjpqzKKEnvgAiVQAiNgA0wZBhsQpArQrAAAAAEgAE1qAAyAAAYAAQdgABNgBIHw AUeQC82ACIXQCrXQHsahIWBaTlcYtow1c5WWC9ITXe40EK1gcQZRaWPbtjDXVnW4EInVcQ3x CmfrEI6VFX5ll/3KVzBBVynbEKQpdWxrSyHIYHCrr99YnjY3EXlUEtvwBiRQAjEQAwVgAUdw BB7QAApgAAAgAAEQAET4ABHAAAaAAAhwABqABIHQAU/gCF7ZCpxwJsKAC8jAby/EDa2xZ2Jq XzPQC3fkFCr1iVERPXdHshOxuyBRDW8QA5brANKrARLQAKD+a7QBgK0GEAEX4LQHoK1iMAge 0AGFsAntwVW10AqdwFXNcAxipRoVmmGD91jDOxTbNKFdUWEIphHMCxLY4AbSqwMOIKREWLSj KwAPAAGmGwGnKwEcUAZlsAEdwAit8CgO12aisApeU53dKB30W7/oYoVScRSRYAELoAMCTACh O7qjiwACYAAP8AAbEAVmIAhHEAFhgAgNZwyk1XvN4JzWEAzrJhIgXMQfkhTBgAYNQAAXgANK +gAIMAAIAMUT8AFhIAdlIAUScASGgAowowvHgA3M0ITCwG+8MMREbMRqbB9VkQpjIAEIrAFC MARDAAZfMAZjEAZDoAEIwAGHYAr+rQApqwDG2LMKkaA1HlVHa7zIq/EVxLAIYJABCcDCDCAA E5ABIGAGj6Cl7eEjkSAKyKALznAKtcDBU8jIqOwYkUsV1bALjtAIhlBwjjALwnAMzMAMwnDL TeQLvnDLYYUUqRzMjNG/imEVwnzMgVHMyozMzIwXylzMzRzNb0HM3LALtgcI7lsV2fAbAlMS pKAFRYWu0jzOmnITfjC6BzC6jVAVyyACJuDOekASemACp3fK5HzPWnETjhAAYHAkxJABAbAL 0NAIvNAIf/AJHoENjgAInmAU0tAIs9AIgKBC0WACcbAMViACRrENpaAHl5A5sZAFJjAJ9TwS +HzSWXH+E5LcF9AQAGSQCgGQzqNLBryLAVEaAB+gDb7AwqPrC8tgAntgDUGwA9yQDRndAyYS DVpgAkwNDPaM0lBNFMR8AGAAEhjwATD9Ac+gDUgQAL2ABAlADNDwAQFwCMQQABlQDYZQ1hX9 ziYQPAoUC9aw1F6wDXlgAjf41FG91z+BEwhgBCDx1zCN0NwwCwHQCAlgBGKA1o6wDWf9B9yw 038QDSiwBZawBSJAClsQBHmAAj0gCZkzz8vwtXxd2i6xyiNBBgGQCh6x1osA04bQEY0QAJ8A 0B8wC9wACH/w2JEdAJMtAvHMDSKwGyYQBJ3ADZeQBtwwzyVt0qb93DJBzc7+MMlkcAhdnQHa ANMHcAiGcAAJQA2L4NKfsNiLINnc8NgVrQWJIgJ5IAsikAWlkAcioNx33dxpDN343RI6AQ1g gM5kcA3cANNGkM4YEHG5PboJsAjn7dsLHghtPSpVUA3cYAlVkgcdoQciYN8dnN8czr9HsQ3P QB8w7QnYAA0goQ3MkBjb4AxSuA3QoA3bYAwy3tzbsAz0YQ0avuEdvuPU8RUj/sx6zeNC7hBg 8QypkMhALpZDvuQOgcZJzhZMHuXA+uTQLOVWXhBUvsxXvuXTkOVVzuVS7uVfDuZLjtpiHhZk HuVnrhhpvuTUvOZ1QSdOzg1OTh9oTMxDvCm7e+czdI4TeT4Sc04Sm0IfCPTnIBHoHRHoc75u FYPoJw7ogh4Sez4SNrHokP4RdR4Sjg7nfhEQADs= --------------30261EB0D60254F9C3F88F7C-- --------------7B9C862AD919005A59FAF245--