spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "holdenk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-18405) Add yarn-cluster mode support to Spark Thrift Server
Date Fri, 25 Nov 2016 14:02:58 GMT

    [ https://issues.apache.org/jira/browse/SPARK-18405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15695936#comment-15695936
] 

holdenk commented on SPARK-18405:
---------------------------------

Even in cluster mode you could overwhelm the node running the thriftserver interface, or are
you proposing to expose the thrifserver interface on multiple nodes not just the driver program?

> Add yarn-cluster mode support to Spark Thrift Server
> ----------------------------------------------------
>
>                 Key: SPARK-18405
>                 URL: https://issues.apache.org/jira/browse/SPARK-18405
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 1.6.0, 1.6.2, 2.0.0, 2.0.1
>            Reporter: Prabhu Kasinathan
>              Labels: Spark, ThriftServer2
>
> Currently, spark thrift server can run only on yarn-client mode. 
> Can we add Yarn-Cluster mode support to spark thrift server?
> This will help us to launch multiple spark thrift server with different spark configurations
and it really help in large distributed clusters where there is requirement to run complex
sqls through STS. With client mode, there is a chance to overload local host with too much
driver memory. 
> Please let me know your thoughts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message