From user-return-11711-apmail-storm-user-archive=storm.apache.org@storm.apache.org Thu Sep 1 12:50:38 2016 Return-Path: X-Original-To: apmail-storm-user-archive@minotaur.apache.org Delivered-To: apmail-storm-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DB72619875 for ; Thu, 1 Sep 2016 12:50:37 +0000 (UTC) Received: (qmail 24670 invoked by uid 500); 1 Sep 2016 12:50:36 -0000 Delivered-To: apmail-storm-user-archive@storm.apache.org Received: (qmail 24623 invoked by uid 500); 1 Sep 2016 12:50:36 -0000 Mailing-List: contact user-help@storm.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@storm.apache.org Delivered-To: mailing list user@storm.apache.org Received: (qmail 24613 invoked by uid 99); 1 Sep 2016 12:50:36 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Sep 2016 12:50:36 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 73C01C05C3 for ; Thu, 1 Sep 2016 12:50:36 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.199 X-Spam-Level: * X-Spam-Status: No, score=1.199 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id hlRtN6CJITWB for ; Thu, 1 Sep 2016 12:50:33 +0000 (UTC) Received: from mail-ua0-f171.google.com (mail-ua0-f171.google.com [209.85.217.171]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id C335B5F479 for ; Thu, 1 Sep 2016 12:50:32 +0000 (UTC) Received: by mail-ua0-f171.google.com with SMTP id i32so142970399uai.2 for ; Thu, 01 Sep 2016 05:50:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=JmvaQ0l7VxVSE48K8crspDhDUn5/T7kBeOmYgnaBRt0=; b=CPWh7kbOtuqmhjzLHb1FE/GHlPN/qcuUDr0lSXRI+odd0ic0sW/WGZoZdEKGUsV8X5 gGRSJYZe7FsaF7sOsEzKW8mBGUoViSH0o3RcyPRmojVwTo9aICu2HciiHJkryu8ra4lv 9n0dUwUrOlHiHm8mt4vwSUsIIStn5SItOjb1l+Fa0yNmW7Kw5YScTpJ78zgb/WPUUr2u xVT+MS1YnHXKjTVH3YP3zFpuq4IMebWr7MCryY74jIyeTXM3VjhoZ5mFryWd4Jy0CcpR 6xhX7xwg5Z5fn1c9oMduITAfVmCyclfPzBudx/Uy+d7TvFdjo7zXYl1FnOTfdASyLz6U CnCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=JmvaQ0l7VxVSE48K8crspDhDUn5/T7kBeOmYgnaBRt0=; b=VrcDKc844K27ZGZwRrtG0NlQAM23K7jvYjhVnXjmfTsxB81OsSMFxlwBQgfx2lRIVG 7sWe+8yhzUrjPoVBefYlhL1OMVtAaAqUoa1e2tGTuOkohhWpV7qCijZrvS1gZa68USJl yED28Us2bja8ZM+ZxAOhWCwgHxxi9qBUoi8LX0oxuDHbQtSVw5k7mK2LQgsEp9tQP35H NeSQK6AqmKcFetGHXNG6goC0Ed9ZKT9Z8P3d7yg39km50tAkOYj+0EeJW930XQ2gomzf vGddLDfN2Jbs7AEzTpsCUMTy3t00vk/AxkHd6o9WCXqJx1xxSEWrRMRiQTwx9gFC0GiM XDAg== X-Gm-Message-State: AE9vXwP2+zyesEio6PE4Y0Uyxv9GM5qQSCiQbzGfJtPk9COEX0kIHUa6TRbkQDAODqRo6mu0QvVg4h5gomYxuA== X-Received: by 10.176.5.67 with SMTP id 61mr8962463uax.35.1472734232055; Thu, 01 Sep 2016 05:50:32 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Noppanit Charassinvichai Date: Thu, 01 Sep 2016 12:50:21 +0000 Message-ID: Subject: Re: How to write unit test if I can't pass object to the bolt? To: user@storm.apache.org Content-Type: multipart/alternative; boundary=94eb2c125112c0162c053b71a8f3 --94eb2c125112c0162c053b71a8f3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable That actually works. I wonder if it's a common pattern for Storm Bolt to do this? IMO, I like to inject things either from the method or constructor. Thanks a lot. On Wed, 31 Aug 2016 at 22:31 Zachary Smith wrote: > Can't you do something like this ala mockito: > > @InjectMocks private KinesisInsertBolt underTest =3D new KinesisInsertBol= t(); > > @Mock private AmazonKinesisClient amazonKinesisClient; > > > On Wed, Aug 31, 2016 at 9:50 PM, Noppanit Charassinvichai < > noppanit.c@gmail.com> wrote: > >> I'm writing a Bolt where a bolt will send data to Kinesis. The library i= s >> AmazonClient. If I initialize AmazonClient outside of prepare() method I >> will get this error >> >> *16:53:32.561* [stderr]Exception in thread "main" java.lang.RuntimeExcep= tion: java.io.NotSerializableException: com.amazonaws.services.kinesis.Amaz= onKinesisClient*16:53:32.561* [stderr] at backtype.storm.utils.Utils.javaSe= rialize(Utils.java:109)*16:53:32.561* [stderr] at backtype.storm.topology.T= opologyBuilder.createTopology(TopologyBuilder.java:107)*16:53:32.561* [stde= rr] at io.conde.data.SparrowFirehoseTopology.main(SparrowFirehoseTopology.j= ava:93)*16:53:32.561* [stderr]Caused by: java.io.NotSerializableException: = com.amazonaws.services.kinesis.AmazonKinesisClient*16:53:32.561* [stderr] a= t java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)*16:= 53:32.561* [stderr] at java.io.ObjectOutputStream.defaultWriteFields(Object= OutputStream.java:1548)*16:53:32.562* [stderr] at java.io.ObjectOutputStrea= m.writeSerialData(ObjectOutputStream.java:1509)*16:53:32.562* [stderr] at j= ava.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)= *16:53:32.562* [stderr] at java.io.ObjectOutputStream.writeObject0(ObjectOu= tputStream.java:1178)*16:53:32.562* [stderr] at java.io.ObjectOutputStream.= writeObject(ObjectOutputStream.java:348)*16:53:32.562* [stderr] at backtype= .storm.utils.Utils.javaSerialize(Utils.java:105) >> >> >> >> So I have to do it like this to not get the Exception >> >> @Override >> public void prepare(Map map, TopologyContext topologyContext, OutputColl= ector outputCollector) { >> this.amazonKinesisClient =3D new AmazonKinesisClient(new CustomCrede= ntialsProviderChain(), new ClientConfiguration()); >> this.random =3D new Random(); >> this.outputCollector =3D outputCollector; >> this.decoder =3D Charset.forName("UTF-8").newDecoder(); >> } >> >> But then I won't be able to inject that and unit test that properly. How >> do I overcome this? I'm using Storm 0.10 >> >> >> > --94eb2c125112c0162c053b71a8f3 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
That actually works. I wonder if it's a common pattern= for Storm Bolt to do this? IMO, I like to inject things either from the me= thod or constructor.

Thanks a lot.

On Wed, 31 Aug 2016 at 22:31 Zachary = Smith <tud20772@temple.edu>= ; wrote:
Can't= you do something like this ala mockito:
@InjectMocks private KinesisInsertBolt underTest =3D new KinesisInsertBolt();

@Mock private AmazonKinesisCl=
ient amazonKinesisClient;

On Wed, Aug 31, 2016 at 9:50 PM, Noppanit Charassinvichai <no= ppanit.c@gmail.com> wrote:
=
I'm writing a Bolt where a bolt will send data to Kine= sis. The library is AmazonClient. If I initialize AmazonClient outside of p= repare() method I will get this error

16:53:32.561 [stderr]Exception in thread "main" java.lang.RuntimeExce=
ption: java.io.NotSerializ=
ableException: com.amazonaws.services.kinesis.AmazonKinesisClient
16:53:32.561 [stderr]	at backtype.storm.utils.Utils.jav=
aSerialize(Utils.java:109)
16:53:32.561 [stderr]	at backtype.storm.topology.Topolo=
gyBuilder.createTopology(TopologyBuilder.java:107)
16:53:32.561 [stderr]	at io.conde.data.SparrowFirehoseT=
opology.main(SparrowFirehoseTopology.java:93)
16:53:32.561 [stderr]Caused by: java.io.NotSerializableException: com.amazonaws.s=
ervices.kinesis.AmazonKinesisClient
16:53:32.561 [stderr]	at java.io.ObjectOutputStream.wri=
teObject0(ObjectOutputStream.java:1184)
16:53:32.561 [stderr]	at java.io.ObjectOutputStream.def=
aultWriteFields(ObjectOutputStream.java:1548)
16:53:32.562 [stderr]	at java.io.ObjectOutputStream.wri=
teSerialData(ObjectOutputStream.java:1509)
16:53:32.562 [stderr]	at java.io.ObjectOutputStream.wri=
teOrdinaryObject(ObjectOutputStream.java:1432)
16:53:32.562 [stderr]	at java.io.ObjectOutputStream.wri=
teObject0(ObjectOutputStream.java:1178)
16:53:32.562 [stderr]	at java.io.ObjectOutputStream.wri=
teObject(ObjectOutputStream.java:348)
16:53:32.562 [stderr]	at backtype.storm.utils.Utils.jav=
aSerialize(Utils.java:105)


So I have to do it like this t= o not get the Exception

=
@Override
public void prepare(Map map, To=
pologyContext topologyContext, OutputCollector outputCollector) {
    this.amazonKinesisClien=
t =3D new AmazonKinesis=
Client(new CustomCredentialsPr=
oviderChain(), new ClientConfi=
guration());
    this.random =3D new Rand=
om();
    this.out=
putCollector =3D outputCollector;
    this.decoder =3D Charset.forName(&quo=
t;UTF-8").newDecoder();}
But then I won't be able to inject that and = unit test that properly. How do I overcome this? I'm using Storm 0.10



--94eb2c125112c0162c053b71a8f3--