ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: After setting streamer and ignite,getting NULL
Date Sun, 27 Sep 2015 20:47:18 GMT
On Sun, Sep 27, 2015 at 1:43 PM, chandresh pancholi <
chandreshpancholi007@gmail.com> wrote:

> Dmitriy,
> In storm, topologies run continuously but in our Unit test we are running
> them only for 20sec. and this the way to do it in Storm.
>

I generally would avoid any sleep() calls at all. If every test in our
TeamCity will run for 20 sec, it will take several days for a single run to
complete.

Can we try to find a way, maybe by listening to some events, to start and
stop all necessary topologies only for the duration of the test?


>
>
> On Mon, Sep 28, 2015 at 2:08 AM, Dmitriy Setrakyan <dsetrakyan@apache.org>
> wrote:
>
> > On Sun, Sep 27, 2015 at 1:36 PM, chandresh pancholi <
> > chandreshpancholi007@gmail.com> wrote:
> >
> > > Gian,
> > > I have checked the code. It looks good.
> > > Just one little change make it  20 sec.
> > >
> > > /* Topology will run for 10sec */
> > > Utils.sleep(20000);
> > >
> >
> > Hm... Can someone explain what is this sleep() for?
> >
> >
> > >
> > >
> > >
> > > On Sun, Sep 27, 2015 at 4:30 PM, Gianfranco Murador <
> > > murador.gianfranco@gmail.com> wrote:
> > >
> > > > Chandresh,  we're ready to do a pull request. Look here:
> > > > https://github.com/murador/ignite/tree/IGNITE-429/modules/storm
> > > >
> > > > 2015-09-27 8:49 GMT+02:00 chandresh pancholi <
> > > > chandreshpancholi007@gmail.com
> > > > >:
> > > >
> > > > > Here is the log file.
> > > > >
> > > > > Thanks
> > > > >
> > > > > On Sat, Sep 26, 2015 at 10:40 PM, Gianfranco Murador <
> > > > > murador.gianfranco@gmail.com> wrote:
> > > > >
> > > > >> Chan,
> > > > >>  I can run the test succefully, but sometimes I get this error:
> > > > >> Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed
> to
> > > add
> > > > >> node to topology because remote node is configured to use loopback
> > > > >> address,
> > > > >> but local node is not (consider changing 'localAddress'
> > configuration
> > > > >> parameter).
> > > > >>
> > > > >> can you send the log file?
> > > > >> Regards, Gianfranco
> > > > >>
> > > > >>
> > > > >>
> > > > >> 2015-09-26 17:13 GMT+02:00 chandresh pancholi <
> > > > >> chandreshpancholi007@gmail.com>:
> > > > >>
> > > > >> > Gian,
> > > > >> > Its still giving "Ouch! Argument cannot be null: streamer"
> > > > >> >
> > > > >> > On Sat, Sep 26, 2015 at 7:24 PM, Gianfranco Murador <
> > > > >> > murador.gianfranco@gmail.com> wrote:
> > > > >> >
> > > > >> > > Chan,
> > > > >> > >  I've fixed some issue, please take a look here:
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://github.com/murador/ignite/blob/IGNITE-429/modules/storm/src/test/java/org/apache/ignite/stream/storm/StormIgniteStreamerSelfTest.java
> > > > >> > >
> > > > >> > > 2015-09-26 8:29 GMT+02:00 chandresh pancholi <
> > > > >> > > chandreshpancholi007@gmail.com
> > > > >> > > >:
> > > > >> > >
> > > > >> > > > Gian,
> > > > >> > > >
> > > > >> > > > I have followed instructions given by you. I instantiate
> > > > >> > > IgniteDateStreamer
> > > > >> > > > and ignite in prepare() method of bolt. but getting
"*Cache
> > > > doesn't
> > > > >> > > > exist*".
> > > > >> > > >
> > > > >> > > > Code snippet from stormStreamer class in prepare()
method
> > > > >> > > >
> > > > >> > > > StormStreamer<String, String, String> stormStreamer
= null;
> > > > >> > > >
> > > > >> > > > Ignite ignite = Ignition.start();
> > > > >> > > >
> > > > >> > > > System.out.println(" ignite ====== "+ignite.toString());
> > > > >> > > >
> > > > >> > > > try(IgniteDataStreamer<String, String> stmr
=
> > > > >> > > > ignite.dataStreamer("cache-0")) {
> > > > >> > > >
> > > > >> > > >     stormStreamer = new StormStreamer<>();
> > > > >> > > >
> > > > >> > > >     IgniteCache<String, String> cache =
> > ignite.cache("cache-0");
> > > > >> > > >
> > > > >> > > >     stmr.allowOverwrite(true);
> > > > >> > > >
> > > > >> > > >     /*  Set ignite instance */
> > > > >> > > >     stormStreamer.setIgnite(ignite);
> > > > >> > > >
> > > > >> > > >     /* Set streamer instance */
> > > > >> > > >     stormStreamer.setStreamer(stmr);
> > > > >> > > >
> > > > >> > > >     /* set thread count */
> > > > >> > > >     setThreads(5);
> > > > >> > > >
> > > > >> > > > }
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Thu, Sep 24, 2015 at 4:19 PM, Anton Vinogradov
<
> > > > >> > > > avinogradov@gridgain.com>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > > Chandresh,
> > > > >> > > > > As far as understand IBolt implementation
should setup all
> > > > >> external
> > > > >> > > > > connections at .prepare() method.
> > > > >> > > > > So, better way is to get existing Ignite
instance or
> create
> > > new
> > > > at
> > > > >> > > > > .prepare()
> > > > >> > > > > method.
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > On Thu, Sep 24, 2015 at 10:55 AM, Gianfranco
Murador <
> > > > >> > > > > murador.gianfranco@gmail.com> wrote:
> > > > >> > > > >
> > > > >> > > > > > Chandresh,
> > > > >> > > > > > I will launch the unit test later this
week , if the
> > problem
> > > > >> > > persists.
> > > > >> > > > I
> > > > >> > > > > > think that you should instantiate Ignite
through the
> > start()
> > > > >> method
> > > > >> > > of
> > > > >> > > > > > Ignition class
> > > > >> > > > > > and retrieve the Ignite cache specifying
a name for the
> > > > default
> > > > >> > > cache.
> > > > >> > > > > You
> > > > >> > > > > > should provide a name also for data
streamer, not
> passing
> > > > null.
> > > > >> > > > > > Could you do this and run again the
unit test ?
> > > > >> > > > > > Thank you,
> > > > >> > > > > > Regards,
> > > > >> > > > > >  Gianfranco
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > 2015-09-24 5:40 GMT+02:00 chandresh
pancholi <
> > > > >> > > > > > chandreshpancholi007@gmail.com
> > > > >> > > > > > >:
> > > > >> > > > > >
> > > > >> > > > > > > Anton,
> > > > >> > > > > > >
> > > > >> > > > > > > Clone this repo
> > > > https://github.com/chandresh-pancholi/ignite
> > > > >> and
> > > > >> > > run
> > > > >> > > > > > this
> > > > >> > > > > > > class
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://github.com/chandresh-pancholi/ignite/blob/master/modules/storm/src/test/java/org/apache/ignite/stream/storm/IgniteStormStreamerSelfTestSuite.java
> > > > >> > > > > > >
> > > > >> > > > > > > You will get the Error saying "Oouch,Argument
is Null"
> > for
> > > > >> > > > > getStreamer().
> > > > >> > > > > > >
> > > > >> > > > > > > On Wed, Sep 23, 2015 at 9:46 PM,
Anton Vinogradov <
> > > > >> > > > > > > avinogradov@gridgain.com>
> > > > >> > > > > > > wrote:
> > > > >> > > > > > >
> > > > >> > > > > > > > Chandresh,
> > > > >> > > > > > > > I'm trying to analize problem.
> > > > >> > > > > > > > Could you please provide more
details and explain
> > > > >> step-by-step
> > > > >> > > how
> > > > >> > > > > can
> > > > >> > > > > > I
> > > > >> > > > > > > > reproduce problem?
> > > > >> > > > > > > >
> > > > >> > > > > > > > On Tue, Sep 22, 2015 at 11:06
PM, chandresh
> pancholi <
> > > > >> > > > > > > > chandreshpancholi007@gmail.com>
wrote:
> > > > >> > > > > > > >
> > > > >> > > > > > > > > Not yet.
> > > > >> > > > > > > > > Still doing brainstorming
why StormStreamer object
> > is
> > > > >> getting
> > > > >> > > > NULLL
> > > > >> > > > > > > > > in-spite setting up ignite
and streamer instances.
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > On Tue, Sep 22, 2015
at 7:22 PM, Vishal Garg <
> > > > >> > > gargvish@gmail.com
> > > > >> > > > >
> > > > >> > > > > > > wrote:
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > > Did you figure it
out? Something related to your
> > > storm
> > > > >> > > > > integration?
> > > > >> > > > > > > > > > Vishal
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > Sent from my iPhone
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > > On Sep 22,
2015, at 4:50 AM, chandresh
> pancholi
> > <
> > > > >> > > > > > > > > > chandreshpancholi007@gmail.com>
wrote:
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > Problem here
is with Storm. Its making
> streamer
> > > > object
> > > > >> > null
> > > > >> > > > > once
> > > > >> > > > > > it
> > > > >> > > > > > > > > goes
> > > > >> > > > > > > > > > > into submit
topology.
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > On Mon, Sep
21, 2015 at 7:13 PM, Lalit Kumar
> > Jha <
> > > > >> > > > > > > > lalitj.dev@gmail.com
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > > wrote:
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > >> Hi Chandresh,
> > > > >> > > > > > > > > > >>
> > > > >> > > > > > > > > > >> See test
suite class here, its based on
> > > annotations
> > > > >> > > > > > > > > > >>
> > > > >> > > > > > > > > > >>
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://github.com/sylentprayer/ignite/blob/ignite-530/modules/twitter/src/test/java/org/apache/ignite/stream/twitter/IgniteTwitterStreamerTestSuite.java
> > > > >> > > > > > > > > > >>
> > > > >> > > > > > > > > > >> No code
required in class body.
> > > > >> > > > > > > > > > >>
> > > > >> > > > > > > > > > >> On Mon,
Sep 21, 2015 at 6:24 PM, chandresh
> > > > pancholi <
> > > > >> > > > > > > > > > >> chandreshpancholi007@gmail.com>
wrote:
> > > > >> > > > > > > > > > >>
> > > > >> > > > > > > > > > >>> Hi
Gian/Vishal/Lalit,
> > > > >> > > > > > > > > > >>>
> > > > >> > > > > > > > > > >>> You
guys have worked on Streamer integration
> > > with
> > > > >> > > Ignite. I
> > > > >> > > > > am
> > > > >> > > > > > > > > working
> > > > >> > > > > > > > > > on
> > > > >> > > > > > > > > > >>> Storm-ignite
integration.
> > > > >> > > > > > > > > > >>>
> > > > >> > > > > > > > > > >>> I am
setting up setStreamer() and
> setIgnite()
> > in
> > > > >> Test
> > > > >> > > class
> > > > >> > > > > and
> > > > >> > > > > > > > > sending
> > > > >> > > > > > > > > > >>> this
object to setBolt() method.
> > > > >> > > > > > > > > > >>>
> > > > >> > > > > > > > > > >>> When
i try to read via gettreamer() and
> > > > getIgnite()
> > > > >> i
> > > > >> > am
> > > > >> > > > > > getting
> > > > >> > > > > > > > > NULL.
> > > > >> > > > > > > > > > I
> > > > >> > > > > > > > > > >> am
> > > > >> > > > > > > > > > >>> trying
to find out why is it happening but
> No
> > > > Luck.
> > > > >> > > > > > > > > > >>>
> > > > >> > > > > > > > > > >>> If
any one of you find some time and look
> into
> > > the
> > > > >> code
> > > > >> > > and
> > > > >> > > > > > help
> > > > >> > > > > > > me
> > > > >> > > > > > > > > to
> > > > >> > > > > > > > > > >>> resolve
this. It would be great.
> > > > >> > > > > > > > > > >>>
> > > > >> > > > > > > > > > >>> Git
:
> > > > https://github.com/chandresh-pancholi/ignite
> > > > >> > > > > > > > > > >>> Module
: storm
> > > > >> > > > > > > > > > >>>
> > > > >> > > > > > > > > > >>> --
> > > > >> > > > > > > > > > >>> Chandresh
Pancholi
> > > > >> > > > > > > > > > >>> Senior
Software Engineer
> > > > >> > > > > > > > > > >>> Flipkart.com
> > > > >> > > > > > > > > > >>> Email-id:chandresh.pancholi@flipkart.com
> > > > >> > > > > > > > > > >>> Contact:08951803660
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > --
> > > > >> > > > > > > > > > > Chandresh Pancholi
> > > > >> > > > > > > > > > > Senior Software
Engineer
> > > > >> > > > > > > > > > > Flipkart.com
> > > > >> > > > > > > > > > > Email-id:chandresh.pancholi@flipkart.com
> > > > >> > > > > > > > > > > Contact:08951803660
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > --
> > > > >> > > > > > > > > Chandresh Pancholi
> > > > >> > > > > > > > > Senior Software Engineer
> > > > >> > > > > > > > > Flipkart.com
> > > > >> > > > > > > > > Email-id:chandresh.pancholi@flipkart.com
> > > > >> > > > > > > > > Contact:08951803660
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > > > --
> > > > >> > > > > > > Chandresh Pancholi
> > > > >> > > > > > > Senior Software Engineer
> > > > >> > > > > > > Flipkart.com
> > > > >> > > > > > > Email-id:chandresh.pancholi@flipkart.com
> > > > >> > > > > > > Contact:08951803660
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > --
> > > > >> > > > Chandresh Pancholi
> > > > >> > > > Senior Software Engineer
> > > > >> > > > Flipkart.com
> > > > >> > > > Email-id:chandresh.pancholi@flipkart.com
> > > > >> > > > Contact:08951803660
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > --
> > > > >> > Chandresh Pancholi
> > > > >> > Senior Software Engineer
> > > > >> > Flipkart.com
> > > > >> > Email-id:chandresh.pancholi@flipkart.com
> > > > >> > Contact:08951803660
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Chandresh Pancholi
> > > > > Senior Software Engineer
> > > > > Flipkart.com
> > > > > Email-id:chandresh.pancholi@flipkart.com
> > > > > Contact:08951803660
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Chandresh Pancholi
> > > Senior Software Engineer
> > > Flipkart.com
> > > Email-id:chandresh.pancholi@flipkart.com
> > > Contact:08951803660
> > >
> >
>
>
>
> --
> Chandresh Pancholi
> Senior Software Engineer
> Flipkart.com
> Email-id:chandresh.pancholi@flipkart.com
> Contact:08951803660
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message