spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandeep Katta <sandeep0102.opensou...@gmail.com>
Subject Re: [Spark-Core]port opened by the SparkDriver is vulnerable to flooding attacks
Date Wed, 28 Feb 2018 17:42:22 GMT
Yeh monitor is present but for some cases like long running job I found App
master is idle.so it will end up closing the App master’s channel so job
will not be completed.

So needed a mechanism to close only invalid connections .


On Wed, 28 Feb 2018 at 10:54 PM, Marcelo Vanzin <vanzin@cloudera.com> wrote:

> Spark already has code to monitor idle connections and close them.
> That's in TransportChannelHandler.java.
>
> If there's anything to do here, it's to allow all users of the
> transport library to support the "close idle connections" feature of
> that class.
>
> On Wed, Feb 28, 2018 at 9:07 AM, sandeep_katta
> <sandeep0102.opensource@gmail.com> wrote:
> > In case of client mode App Master and Driver are in different JVM
> process,the
> > port opened by the Driver is vulnerable for flooding attacks as it is not
> > closing the IDLE connections.
> >
> > I am thinking to fix this issue using below mechanism
> >
> > 1.Expose configuration to close the IDLE connections as secondary
> developers
> > can decide based on their use case
> >
> > 2.Maintain the SET of validConnections,the condition for
> validConnections is
> > *TransportChannelHandler.channelRead* event will be fired for the valid
> > connections.
> >
> > 3.Whenever there is timeout(no communication happened across the channel
> for
> > x time),close the connection.
> >
> >    if !validConnectionSet then
> >       close the connection
> >
> > Note: Timeout value is configurable using the config
> "spark.network.timeout"
> >
> >
> > can this solution be used to solve this issue,if any suggestions are also
> > welcomed.
> >
> > Regards
> > Sandeep
> >
> >
> >
> > --
> > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> >
>
>
>
> --
> Marcelo
>

Mime
View raw message