storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stig Rohde Døssing <stigdoess...@gmail.com>
Subject Re: How to configure fixed range of ports for (Storm/Netty) internal TCP connections.
Date Mon, 15 Apr 2019 11:53:53 GMT
I'm not sure what this could be.

There are a couple of spots I could see where a random port could be
picked. Are you using
https://github.com/apache/storm/blob/v1.2.1/storm-core/src/jvm/org/apache/storm/metric/HttpForwardingMetricsServer.java?
Are you using the "storm dev-zookeeper" command?

If not, try enabling trace logging for the supervisor (you do this in the
log4j2/cluster.xml file), and see if the port pops up in
logs/supervisor.log. Maybe we can see where it is coming from.

Den man. 15. apr. 2019 kl. 12.49 skrev Thomas Julian <thomasjulian@zoho.com
>:

> Hi Stig,
>
> "It would be good to know which commands those Java processes are running"
>
> It's a supervisor process but not sure about the commands being executed.
>
> **@***.**.**.110~>lsof -nPi -sTCP:LISTEN
>
> COMMAND   PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
>
> *java*    *33536**  ****   *95u*  *IPv6 23496886*      *0t0*  *TCP
> *:38527 (LISTEN)*
>
> java    33536  ***   96u  IPv6 23496887      0t0  TCP *:3337 (LISTEN)
>
> java    42020  ***  118u  IPv6 27777549      0t0  TCP *:6700 (LISTEN)
>
> **@***.**.**.110~>ps auxxx | grep java
>
> **      *33536*  0.1  1.9 6779344 312460 ?      Sl   Apr06  23:51 ....
> *org.apache.storm.daemon.supervisor.Supervisor*
>
> Could you try getting the output of "jps -lvm <pid>" for the Java
> processes that have the ports open?
>
> It's not working.
>
> **@***.**.**.110~>jps -lvm 33536
>
> RMI Registry not available at 33536:1099
>
> Exception creating connection to: 33536; nested exception is:
>
> java.net.SocketException: Invalid argument or cannot assign requested
> address
>
> but this one works. Will it help?
>
> **@***.**.**.110~>jps -l
>
> 33536 org.apache.storm.daemon.supervisor.Supervisor
>
>
>
> Thanks & Regards,
> Julian.
>
> ------------------------------
>
>
> ---- On Mon, 15 Apr 2019 15:00:56 +0530 *Stig Rohde Døssing
> <stigdoessing@gmail.com <stigdoessing@gmail.com>>* wrote ----
>
> It would be good to know which commands those Java processes are running.
> Could you try getting the output of "jps -lvm <pid>" for the Java processes
> that have the ports open?
>
>
> Den man. 15. apr. 2019 kl. 07.21 skrev Thomas Julian <
> thomasjulian@zoho.com>:
>
>
> Hi Roshan,
>
> Thank you for your reply. What I understand from your reply is that Storm
> or its internal components(netty, zeromq etc) won't use any ports or its
> ranges for their TCP communication other than ones(6700, 6701, 6702, 6703)
> specified in storm.yaml? But in a previous deployment, we observed that
> storm uses ports 39855, 35392 to listen for TCP connections. Is this only
> possible, when we don't explicitly configure the port numbers for workers?
>
> > tcp6 0 0 :::39855 :::* LISTEN 9943/java
> > tcp6 0 0 :::35392 :::* LISTEN 9521/java
>
>
> Thanks & Regards,
> Julian
>
>
> ------------------------------
>
>
>
>
>
> ---- On Sun, 14 Apr 2019 12:59:13 +0530 *Roshan Naik
> <roshan_naik@yahoo.com <roshan_naik@yahoo.com>>* wrote ----
>
>
>
> Julian,
>    The config `supervisor.slots.ports` is used to control the number of
> workers on each Supervisor as well as the range ports on which the workers
> should listen for incoming connections. For example, the default config is:
>
> supervisor.slots.ports
>  - 6700
>  - 6701
>  - 6702
>  - 6703
>
> which means 4 workers per supervisor and each worker should bind to and
> listen on one of those 4 ports.
>
> -roshan
>
>
>
>
> On Friday, April 12, 2019, 6:09:37 AM EDT, Thomas Julian <
> thomasjulian@zoho.com> wrote:
>
>
>
>
>
> Hi,
>
> Hope the question is clear. Please someone respond or let me know, if the
> question is not clear.
>
> Thanks & Regards,
> Julian
>
> ________________________________
> ---- On Wed, 03 Apr 2019 13:15:07 +0530 Thomas Julian <
> thomasjulian@zoho.com> wrote ----
>
> > Team,
> >
> > We have provisioned a distributed storm environment(version:1.2.1) and
> noticed that Storm(or netty) uses some random ports to listen for TCP
> connections. Can these port(or ranges) be configured, as we do
> for supervisor.slots.ports:
> >   - 6700
> > Ex.
> >
> > tcp6 0 0 :::39855 :::* LISTEN 9943/java
> > tcp6 0 0 :::35392 :::* LISTEN 9521/java
> >
> > Noted these connections on a supervisor node. We need to configure these
> ports in firewall. Please guide!
> >
> > Thanks & Regards,
> > Julian.
> >
>
>
>
>
>
>

Mime
View raw message