spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rob Vesse (JIRA)" <>
Subject [jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates
Date Fri, 31 Aug 2018 08:13:00 GMT


Rob Vesse commented on SPARK-24434:

I think the miscommunication here was because of the discrepancy between this Jira and k8s-sig-big-data
weekly meeting notes

As an Apache member this comment raises red flags for me.  All Spark development discussions
should either be happening on Apache resources (JIRA, mailing lists, GitHub repos) or being
captured and posted to Apache resources.  If people are having to follow external resources,
particularly live meetings which naturally exclude portions of the community due to timezone/availability
constraints, to participate in an Apache community then that community is not operating as
a proper Apache community.  

This doesn't mean that such discussions and meetings can't happen but they should be summarised
back on Apache resources so the wider community has the opportunity to participate.

> Support user-specified driver and executor pod templates
> --------------------------------------------------------
>                 Key: SPARK-24434
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: Kubernetes
>    Affects Versions: 2.4.0
>            Reporter: Yinan Li
>            Priority: Major
> With more requests for customizing the driver and executor pods coming, the current approach
of adding new Spark configuration options has some serious drawbacks: 1) it means more Kubernetes
specific configuration options to maintain, and 2) it widens the gap between the declarative
model used by Kubernetes and the configuration model used by Spark. We should start designing
a solution that allows users to specify pod templates as central places for all customization
needs for the driver and executor pods. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message