spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anirudh Ramanathan <>
Subject Re: Kubernetes: why use init containers?
Date Fri, 12 Jan 2018 22:09:31 GMT
That's fair - I guess it would be a stretch to assume users wouldn't put
custom logic in their init containers if that hook is provided to them. :)

Experimental sounds like a good idea for 2.3. Gives us enough wriggle room
for the next one, and hopefully user feedback in the meantime.


On Jan 12, 2018 2:00 PM, "Marcelo Vanzin" <> wrote:

> On Fri, Jan 12, 2018 at 1:53 PM, Anirudh Ramanathan
> <> wrote:
> > As I understand, the bigger change discussed here are like the init
> > containers, which will be more on the implementation side than a user
> facing
> > change/behavioral change - which is why it seemed okay to pursue it post
> 2.3
> > as well.
> It's not just a code change.
> There are multiple configurations exposed to control the init
> container. There's a whole step - the running of the init container -
> that currently can be customized (even though there is no
> documentation on how to safely do that). If you ship that as "stable",
> you cannot later change it in way that will break applications. So
> you'd not only be stuck with the existence of the init container, but
> with all its current behavior.
> Marking as experimental gives us time to stabilize these details. Not
> just whether the init container exists, but what is its actual
> behavior and how the use can affect it. A lot of the replies here
> always mention that init containers can be customized, but I just want
> to point out again that there is currently zero documentation about
> how to do that and not break the assumptions the spark-on-k8s
> submission code makes.
> The same applies to the other images, by the way.
> --
> Marcelo

View raw message