Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E1EF0200D49 for ; Fri, 10 Nov 2017 05:42:23 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id E09A6160C02; Fri, 10 Nov 2017 04:42:23 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 0B587160BEF for ; Fri, 10 Nov 2017 05:42:22 +0100 (CET) Received: (qmail 11114 invoked by uid 500); 10 Nov 2017 04:42:22 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 11101 invoked by uid 99); 10 Nov 2017 04:42:21 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Nov 2017 04:42:21 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id F0186182ECF for ; Fri, 10 Nov 2017 04:42:20 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.879 X-Spam-Level: ** X-Spam-Status: No, score=2.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_REPLY=1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id zquUXnyZQ7FL for ; Fri, 10 Nov 2017 04:42:19 +0000 (UTC) Received: from mail-it0-f41.google.com (mail-it0-f41.google.com [209.85.214.41]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 378B15FD1B for ; Fri, 10 Nov 2017 04:42:19 +0000 (UTC) Received: by mail-it0-f41.google.com with SMTP id 72so162278itk.3 for ; Thu, 09 Nov 2017 20:42:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=yvUujbNiW9xJwrclEQ5V+KUNpJGOuvCoyXc67TSKvio=; b=ccdZyhjdC4QH3ucISyu2hCQSQlBipuXSIdTGCaSC29Bymc0qlFFo+v0ovLRS7VCRj8 VRCVx69Nj0ejKFkzkDDkcEPI+2R+2EjFhWVZjtqaZ81QuWCR+dUUxwTr/90S6Ce8JRcW BBz6qL1MtqTUo3IG8OQGHTsgOP5yDv7ioFjXcB68G0rRRu73tdEKOgbItM++1ksMOKZB 6xFn9Q16BMyyM5Vgu2Hat6mKX+QzHJcIxaurRqWsZRRzBEpvvv23vhOL5q23Pip5Onbk wqKogdYFQ+jAba5VbjPqJEWNdxakYkPvtR4KeH0I5qzHBa9YzzrmTgIiwn4XO3fjh8Hn KfkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=yvUujbNiW9xJwrclEQ5V+KUNpJGOuvCoyXc67TSKvio=; b=TDGaafIsfOBbax+59pobmnDOFKM5165Pl2/s0yDYgRQffmjzY7ea50AJtEaJ8NbtXQ eEf8dMBVxDuz2hCRd67sN1FbQRmCDh638RbSZXV5SwRcDVze5VRM7fqIpR8dooxwJWcI rsgc2KQ9IINjurxden4mmkwiIW4M91oP2M+r5b6MmkQN88CNfGViZy6aA1uCNQ9jFlSz HgL/tUTNq06WFYmg/nT9c2kMOeJGNQLQ2gulsF31TWydZ8SY8nhowFhnXCNC0wqTyGSI Lx07JeieUcPwhOIs+b9zpiPNl2mmpKbmgvXiHTDr3rzYMirwTGPuPcHrNqNJz3L5HrmN UDtw== X-Gm-Message-State: AJaThX5evlnYagDpf9EhIt3ATkpLRK4QGhxCZ6zYYGWW3b3pkaTCqUzi 2uniriprh3NROq0ypeCA7Jwp1ObfN3Jv6LZJBoI= X-Google-Smtp-Source: AGs4zMZQIriakEORP7iOpuReWw+XS31oURns/v0ROBIXxctct2YTX+mBrlmuk7VfAY7kkJoKOMnuEEvrUCXi9cd16GE= X-Received: by 10.36.228.68 with SMTP id o65mr130196ith.128.1510288938662; Thu, 09 Nov 2017 20:42:18 -0800 (PST) MIME-Version: 1.0 References: <64921159.4140051.1510105244703.ref@mail.yahoo.com> <64921159.4140051.1510105244703@mail.yahoo.com> <2031048013.4666036.1510175202292@mail.yahoo.com> <852096418.67589.1510288431237@mail.yahoo.com> In-Reply-To: <852096418.67589.1510288431237@mail.yahoo.com> From: Valentin Kulichenko Date: Fri, 10 Nov 2017 04:42:08 +0000 Message-ID: Subject: Re: JMSStreamer enhancement -- needs to support multiple IgniteDataStreamers. To: Timothy Steffens Cc: dev@ignite.apache.org Content-Type: multipart/alternative; boundary="94eb2c124520b2a029055d998bb0" archived-at: Fri, 10 Nov 2017 04:42:24 -0000 --94eb2c124520b2a029055d998bb0 Content-Type: text/plain; charset="UTF-8" Tim, If you want to do this change by yourself, then create a ticket in Jira, assign to yourself and prepare a pull request. See here for more information about the process: https://ignite.apache.org/community/contribute.html#contribute https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute -Val On Thu, Nov 9, 2017 at 8:34 PM Timothy Steffens wrote: > Yes, this makes sense. The proposed changes will improve the overall > implementation of the JMSStreamer and the changes will also allow for me to > then extend the streamer and implement my own custom functionality as > needed. > > What are the next steps to getting this change implemented? > > > > On Thursday, November 9, 2017, 6:11:01 PM CST, Valentin Kulichenko < > valentin.kulichenko@gmail.com> wrote: > > > Tim, > > I think you properly defined flaws in the JmsStreamer and it defintely > makes sense to do the following: > > - Deprecate org.apache.ignite.stream.jms11.MessageTransformer in favor > > of extractors defined in StreamAdapter. This JMS specific transformer > doesn't provide any value and only duplicates generic extractors making > it > very confusing. > - Remove JmsStreamer#processMessage method and use > > StreamAdapter#addMessage. > > This will make both API and implementation cleaner and more in line with > overall streamers design. > > And the best thing: there is no need to add special support for multiple > streamers anymore. If one has one queue/many caches use case, they can > simply extend JmsStreamer and override addMessage method (which is > protected). This way the behavior can be modified in any desired way, while > reusing most of JmsStreamer code. > > Thoughts? > > -Val > > On Wed, Nov 8, 2017 at 1:31 PM, Dmitriy Setrakyan > wrote: > > > On Thu, Nov 9, 2017 at 5:06 AM, Timothy Steffens < > > timothy.steffens@yahoo.com > > > wrote: > > > > > Can the JMSStreamer be configured to inspect the message prior to > picking > > > it off of the queue? If so, then I can use multiple JMSStreamers as you > > > suggest. If not, then I cannot since each JMSStreamer will not know if > > the > > > next message is for them or for another to process. I do not want to > have > > > multiple queues (one queue per table - which would equate to one > > > JMSStreamer per queue) due to ongoing maintenance concerns. > > > > > > > To be honest, queue-per-table seems like the most appropriate solution > > here. I do not think inspecting messages before they get picked off the > > queue will perform well. > > > > If queue-per-table is not possible, then my suggestion would be not to > use > > JMS Streamer at all, and implement your own logic for reading JMS > messages > > and passing them off to the appropriate data streamer. > > > > D. > > > --94eb2c124520b2a029055d998bb0--