storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nicol...@uom.gr
Subject Re: Restart Nimbus Issues
Date Sat, 08 Jun 2019 10:13:50 GMT
Thank you very much for your valuable time and help.
With your help I just realised how transient are nimbus and ui failures in
Storm. My problem is solved.

Best Regards,
Nicole


> Hmm, that supervisord conf has two entries for starting up storm nimbus.
> One in supervisord.conf, and the other in the supervisor.d/ folder. You
> should only need one of those. I would tend to use the
> supervisor.d/storm-nimbus.conf version, and remove the one in
> supervisord.conf. Although I would also use the code from the
> supervisord.conf version... If that makes sense.
> So you just have:
> supervisor.d/storm-nimbus.conf
>
> [program:storm-nimbus]
> command=/usr/local/storm/bin/storm nimbus
> autostart=true
> autorestart=true
> startsecs=10
> startretries=999
> log_stdout=true
> log_stderr=true
> logfile=/var/log/storm/nimbus.out
> logfile_maxbytes=20MB
> logfile_backups=10
>
> And no [program:storm-nimbus] in supervisord.conf
>
> It's possible supervisord ended up running both of them, and could only
> track one. I'm not really sure what it does when it sees duplicate
> sections. (It basically reads all the files in supervisor.d folder and
> appends them to the end of the supervisord.conf config file to get it's
> configuration).
>
> Either way, it seems likely that you had a nimbus process running on the
> box, then a new nimbus processes that supervisord was trying to startup
> was
> failing because the storm nimbus port(s) were already in use. To start a
> topology you have to have a nimbus running (unless you are running things
> in local mode), so it is very unlikely it wasn't running already.
>
> On Linux you can check for running nimbus process from a terminal with:
> ps aux | grep nimbus
> and kill it using the pid returned in the above command and kill command.
>
> Yes you should really of just replied to the storm user group. I read
> everything on there anyway.
> I've sent this reply to the group as well as yourself.
>
> Regards,
>
> Peter
>
> On Fri, 7 Jun 2019 at 12:29, <nicoleta@uom.gr> wrote:
>
>> Hello Peter and thanks for your interest!
>>
>> The code in storm-nimbus.conf (located in conf.d folder in supervisor
>> folder) is as follows:
>>
>>
>> [program:storm-nimbus]
>> command=/usr/local/storm/bin/storm nimbus
>> autorestart=true
>>
>> The code in supervisord.conf (located in supervisor folder)is:
>>
>> [program:storm-nimbus]
>> command=/usr/local/storm/bin/storm nimbus
>> autostart=true
>> autorestart=true
>> startsecs=10
>> startretries=999
>> log_stdout=true
>> log_stderr=true
>> logfile=/var/log/storm/nimbus.out
>> logfile_maxbytes=20MB
>> logfile_backups=10
>>
>> However, I encounter the same problem when running the nimbus deamon
>> manually (bin/storm nimbus). It tries to start and exits immediately.
>> Nevertheless, I can run and kill topologies with no problem at all.
>>
>> I finally decided to destroy my master node and build it again. Now
>> everything wokrs fine and the deamon does not exit. It seems to run
>> properly. Which is the most appropriate way to terminate a deamon and
>> stop
>> its function (for real, not as I did before...)?
>>
>> Thank you in advance.
>>
>> Regards,
>> Nicole
>>
>> PS. I am sorry for using your personal email address. Should I just
>> answer
>> the original email and send it back to user@storm.apache.org?
>>
>>
>>
>> > It sounds to me, like the command to start nimbus in the supervisord
>> > configuration is launching it via another process. And the other
>> process
>> > is
>> > what is getting it's PID tracked by supervisord.
>> > What is the supervisord configuration like for nimbus?
>> >
>> > On Thu, 6 Jun 2019 at 15:36, Stig Rohde D?ssing
>> <stigdoessing@gmail.com>
>> > wrote:
>> >
>> >> Hi Nicole,
>> >>
>> >> No, this is the right place.
>> >>
>> >> Your question is a little unclear to me since it is hard to tell
>> whether
>> >> you're talking about the supervisor Storm daemon, or the supervisord
>> >> tool
>> >> for running services. I'm going to assume you are asking why
>> >> storm-nimbus
>> >> is in the BACKOFF state, and whether it is still running.
>> >>
>> >> Going by the supervisord docs
>> (http://supervisord.org/subprocess.html),
>> >> it sounds like supervisord tried to start Nimbus, decided it didn't
>> >> succeed, and placed it into the BACKOFF state. This is a temporary
>> >> waiting
>> >> state where supervisord will wait a bit, before trying to start
>> Nimbus
>> >> again. Nimbus may or may not be running when in this state.
>> Supervisord
>> >> may
>> >> have assumed wrong, and Nimbus actually managed to start. In your
>> case,
>> >> Nimbus is clearly running since you can submit topologies.
>> >>
>> >> You can most likely figure out whether Nimbus crashed, and why, by
>> >> reading
>> >> the Nimbus log. The default location is in
>> >> your-storm-dir/logs/nimbus.log.
>> >>
>> >> Den tor. 6. jun. 2019 kl. 12.31 skrev <nicoleta@uom.gr>:
>> >>
>> >>> Hello,
>> >>>
>> >>> I am Nicole and I am a researcher in University of Macedonia,
>> Greece. I
>> >>> am
>> >>> new to Storm and I found this email in the "Getting Help" Section of
>> >>> Storm's site.
>> >>>
>> >>> I started using Storm a few days ago and I have a question on
>> Nimbus. I
>> >>> tried to stop (using both ways i.e Ctrl-C and supervisor) the
>> Nimbus,
>> >>> UI
>> >>> and Supervisor deamons and start them again and I had some issues.
>> When
>> >>> I
>> >>> tried to restart Nimbus it seems to exit fast on its own (like
>> pressing
>> >>> Ctrl-C). When I tried to do the same with supervisor I got the
>> >>> following
>> >>> results:
>> >>>
>> >>> # sudo supervisorctl status
>> >>> storm-nimbus    BACKOFF   Exited too quickly (process log may have
>> >>> details)
>> >>> storm-ui        RUNNING   pid 4024, uptime 15:43:28
>> >>>
>> >>> Nevertheless, Nimbus seems to run normally as I can run a new
>> topology
>> >>> and
>> >>> kill it with no problem at all. Why does this happen? Is my Nimbus
>> >>> still
>> >>> on?
>> >>>
>> >>> In case this is not the right place to ask such questions I am sorry
>> >>> for
>> >>> the inconvenience. Is there a community to ask questions? Thank you
>> >>> very
>> >>> much in advance.
>> >>>
>> >>> Regards,
>> >>> Nicole
>> >>>
>> >>>
>> >
>> > --
>> >
>> >
>> >
>> > *Peter Chamberlain* | Senior Software Engineer | HTK
>> >
>> > T: +44(0)870 600 2311
>> > Connect with me: Email <peter.chamberlain@htk.co.uk>
>> >
>> >
>> > [image: htk logo] <http://www.htk.co.uk/>
>> >
>> > Connect with HTK: htk.co.uk <http://www.htk.co.uk/> | LinkedIn
>> > <http://www.linkedin.com/company/htk/> | Twitter
>> > <http://www.twitter.com/htkhorizon>
>> >
>> > HTK Limited, Chapmans Warehouse, Wherry Quay, Ipswich, IP4 1AS, UK.
>> > Company Registered in England and Wales as 3191677, VAT Number 675
>> 9467
>> 71
>> >
>> >
>> > PLEASE CONSIDER THE ENVIRONMENT BEFORE PRINTING THIS EMAIL.
>> > This email is only for the use of the intended recipients and may
>> contain
>> > privileged information. If you’ve received this email in error, please
>> let
>> > the sender know; then delete the message. The views expressed in this
>> > email
>> > represent those of the sender and not necessarily of HTK.
>> >
>>
>>
>>
>
> --
>
>
>
> *Peter Chamberlain* | Senior Software Engineer | HTK
>
> T: +44(0)870 600 2311
> Connect with me: Email <peter.chamberlain@htk.co.uk>
>
>
> [image: htk logo] <http://www.htk.co.uk/>
>
> Connect with HTK: htk.co.uk <http://www.htk.co.uk/> | LinkedIn
> <http://www.linkedin.com/company/htk/> | Twitter
> <http://www.twitter.com/htkhorizon>
>
> HTK Limited, Chapmans Warehouse, Wherry Quay, Ipswich, IP4 1AS, UK.
> Company Registered in England and Wales as 3191677, VAT Number 675 9467 71
>
>
> PLEASE CONSIDER THE ENVIRONMENT BEFORE PRINTING THIS EMAIL.
> This email is only for the use of the intended recipients and may contain
> privileged information. If you’ve received this email in error, please let
> the sender know; then delete the message. The views expressed in this
> email
> represent those of the sender and not necessarily of HTK.
>


Mime
View raw message