From user-return-14787-archive-asf-public=cust-asf.ponee.io@storm.apache.org Thu Sep 24 11:30:16 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-he-de.apache.org (mxout1-he-de.apache.org [95.216.194.37]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id E6EE718063D for ; Thu, 24 Sep 2020 13:30:15 +0200 (CEST) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-he-de.apache.org (ASF Mail Server at mxout1-he-de.apache.org) with SMTP id 74AE664874 for ; Thu, 24 Sep 2020 11:30:14 +0000 (UTC) Received: (qmail 31480 invoked by uid 500); 24 Sep 2020 11:30:12 -0000 Mailing-List: contact user-help@storm.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@storm.apache.org Delivered-To: mailing list user@storm.apache.org Received: (qmail 31470 invoked by uid 99); 24 Sep 2020 11:30:12 -0000 Received: from spamproc1-he-fi.apache.org (HELO spamproc1-he-fi.apache.org) (95.217.134.168) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 24 Sep 2020 11:30:12 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-fi.apache.org (ASF Mail Server at spamproc1-he-fi.apache.org) with ESMTP id EED9EC0104 for ; Thu, 24 Sep 2020 11:30:11 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-fi.apache.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamproc1-he-fi.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([116.203.227.195]) by localhost (spamproc1-he-fi.apache.org [95.217.134.168]) (amavisd-new, port 10024) with ESMTP id TxHm0Y0S6ofj for ; Thu, 24 Sep 2020 11:30:11 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::d2c; helo=mail-io1-xd2c.google.com; envelope-from=rui.abreu@gmail.com; receiver= Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 0045F7F981 for ; Thu, 24 Sep 2020 11:30:10 +0000 (UTC) Received: by mail-io1-xd2c.google.com with SMTP id r25so2951172ioj.0 for ; Thu, 24 Sep 2020 04:30:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=7fcX12siKKyF2WVN0UAmr2f03Lm6MF5/vX+N7F4/IQ8=; b=Ub6rXZxHga4ftIuFu+wFz5y3d+D+Jx9fNN7jIvwMjMsyE/YsL5O9tRniIU2U/hizrj EcK+fTXbfc8J5tTgeaqDRSLRFuulFKxBwUotjFEeKS8w/gjNZlghJylC7eDNJIqh7bKW c9L1wlB+7ZmCMjEbGdNwMDANdjmiXz+LYN7C6yMAiWVTKVRZHZTad6XTJoBOOhxC6xdp AHmKYmtTkzMSwReTMbe7p+R31iqp5ATM16Z1EPMPSYlydGULOBXCC8MNJvQ08ny3yG4R tIC9vAqjFU7kIcaqSMkO5yEbwjxtOhCPG6LG+f7PWBP/4pRWInS6zEA2+8uXikxLvtd5 ix5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=7fcX12siKKyF2WVN0UAmr2f03Lm6MF5/vX+N7F4/IQ8=; b=YrOXyRuAfRjANzFjJts+X8arB1cM56kFvHjRndHGlseKTTvtO56NX6FYUoah+VJVqR l4ItYgKbjyPyn6/MWmih7UlWmysu84chYMFDZhiuoIt1AZOtt9gDvppQjQ1e/9jtx85Z oqCCG9MgglXyz6DujIo121vwJBN0YKJd6BRzTDXpI9VeAaI2CaD4J8TQBErzEtftFlpl 4R3B4gL9/tLrPFYVxqNjgs3PWFwPUSOZMcpRxsCAIdpP9FAhhZ2+XCAKKQ9uLl3eDSQe nrewJyuP7JiNDSmD1mSbKSK6ACfBZ+Kqfuy6t6CHtxL6Rv02UrQj1bThJUB3FiZsuKvT vOlg== X-Gm-Message-State: AOAM532v+eX965jZ3snhWMNzNy0g4h4guYDZBuvrAtB8ZMsZc/Zd+7Vy nL5/0VmaT+L3pkC2xPr6mKEf56CjKBvay44/AAqanzxgK/Caog== X-Google-Smtp-Source: ABdhPJw7kRlIj/GDHh5hvAEAZHdiRGP3r1d1QbKa79iTsUbRfpsfegTC6yQJTupxuiPLy7wXOmJb1IKQRf+5+L6EBk0= X-Received: by 2002:a5e:a705:: with SMTP id b5mr2874481iod.73.1600947009443; Thu, 24 Sep 2020 04:30:09 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Rui Abreu Date: Thu, 24 Sep 2020 12:29:43 +0100 Message-ID: Subject: Re: Scheme in Spouts To: user@storm.apache.org Content-Type: multipart/alternative; boundary="000000000000cd5eae05b00d856e" --000000000000cd5eae05b00d856e Content-Type: text/plain; charset="UTF-8" Hi Paul, The class still exists under a new package: https://storm.apache.org/releases/current/javadocs/org/apache/storm/spout/SchemeAsMultiScheme.html On Thu, 24 Sep 2020 at 12:14, Paul Jose wrote: > > > Hi, > > My name is Paul. I am tasked with upgrading a storm topology running on > the 0.10.0 version to 2.2.0. > Flux was used for the configuration. > This is the configuration used for the spout in 0.10.0. > > components: > > - className: com.platform.metric.spouts.SimpleJSONScheme > id: jsonScheme > > - className: backtype.storm.spout.SchemeAsMultiScheme > constructorArgs: > - > ref: jsonScheme > id: stringMultiScheme > > - className: storm.kafka.ZkHosts > constructorArgs: > - "${kafka.zk.host}" > id: zkHosts > > - className: storm.kafka.SpoutConfig > constructorArgs: > - > ref: zkHosts > - "${kafka.topic}" > - "" > - stormKafka1 > id: kafkaSpoutConfig > properties: > - > name: ignoreZkOffsets > value: false > - > name: "startOffsetTime" > value: -1 > - > name: scheme > ref: stringMultiScheme > - > name: fetchSizeBytes > value: 8388608 > > I've updated most of the topology to 2.2.0 but I can't seem to find any > mention of schemes in 2.2.0. > Can somebody point me in the right direction on how to upgrade. Do I have > to use the Deserialiser class and apply the implmentation there and then > pass it as teh value for the "value.deserialiser" property of the spout? > > Any direction or suggestion would be really helpful. > > Best Regards, > > *Paul Jose* > ---------------------------------------------------------------------------------------Disclaimer---------------------------------------------------------------------------------------------- > > ****Views and opinions expressed in this e-mail belong to their author and > do not necessarily represent views and opinions of Ugam. Our employees are > obliged not to make any defamatory statement or infringe any legal right. > Therefore, Ugam does not accept any responsibility or liability for such > statements. The content of this email is confidential and intended for the > recipient specified in message only. It is strictly forbidden to share any > part of this message with any third party, without a written consent of the > sender. If you have received this message by mistake, please reply to this > message and follow with its deletion, so that we can ensure such a mistake > does not occur in the future. Warning: Sufficient measures have been taken > to scan any presence of viruses however the recipient should check this > email and any attachments for the presence of viruses as full security of > the email cannot be ensured despite our best efforts. Therefore, Ugam > accepts no liability for any damage inflicted by viewing the content of > this email.. **** > Please do not print this email unless it is necessary. Every unprinted > email helps the environment. > > --000000000000cd5eae05b00d856e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
<= div dir=3D"ltr" class=3D"gmail_attr">On Thu, 24 Sep 2020 at 12:14, Paul Jos= e <paul.j2@ugamsolutions.co= m> wrote:


=20
Hi,

My name is Paul. I am tasked with upgrading a storm topology running on the= 0.10.0 version to 2.2.0.
Flux was used for the configuration.=C2=A0
This is the configuration used for the spout in 0.10.0.
components:

- className: com.platform.metric.spouts.SimpleJSONScheme
= id: jsonScheme

- className: backtype.storm.spout.S= chemeAsMultiScheme
constructor= Args:
-
re= f: jsonScheme
id= : stringMultiScheme

- classN= ame: storm.kafka.ZkHosts
constructorArgs:
- = "${kafka.zk.host}"
id: zkHosts
- className: storm.kafka.S= poutConfig
constructorArgs
:
-
ref= : zkHosts
- "${kafka.to= pic}"
- <= span style=3D"color:rgb(106,135,89)">""
- stormKafka1
id: kafkaSpoutConfig
properties:
-
name: ignoreZkOffsets
value: false
-
name: "startOffsetTime"
value:= -1
-
name= : scheme
ref: strin= gMultiScheme
-
na= me: fetchSizeBytes
= value: 8388608
I've updated most of the topology to 2.2.0 but I can't seem to find= any mention of schemes in 2.2.0.
Can somebody point me in the right direction on how to upgrade. Do I have t= o use the Deserialiser class and apply the implmentation there and then pas= s it as teh value for the "value.deserialiser" property of the sp= out?

Any direction or suggestion would be really helpful.

Best Regards= ,

Paul Jose

---------------------------------------------------------------------------= ------------Disclaimer-----------------------------------------------------= -----------------------------------------=20
****Views and opinions expressed in this e-mail belong to their author and= do not necessarily represent views and opinions of Ugam.=20 Our employees are obliged not to make any defamatory statement or infringe = any legal right.=20 Therefore, Ugam does not accept any responsibility or liability for such st= atements. The content of this email is confidential and intended for the re= cipient specified in message only. It is strictly forbidden to share any pa= rt of this message with any third party, without a written consent of the s= ender. If you have received this message by mistake, please reply to this message = and follow with its deletion, so that we can ensure such a mistake does not= occur in the future.=20 Warning: Sufficient measures have been taken to scan any presence of viruse= s however the recipient should check this email and any attachments for the= presence of viruses as full security of the email cannot be ensured despit= e our best efforts. Therefore, Ugam accepts no liability for any damage inflicted by viewing th= e content of this email.. ****
Please do not print this email unless it is necessary. Every unprinted emai= l helps the environment.=20

=20
--000000000000cd5eae05b00d856e--