openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dragos Dascalita Haut <ddas...@adobe.com.INVALID>
Subject Re: Invoker HA on Mesos
Date Fri, 30 Mar 2018 17:22:15 GMT
"... Interacting with the Kubernetes API server to do a labeling operation takes around 10ms,
so we
couldn't do this on a truly hot path...."


I was gonna make a similar comment for this "data plane" part, imagining that a performance
optimized LB would be interested to have this visibility in memory, w/o relying on other components
for each single invocation.

________________________________
From: David P Grove <groved@us.ibm.com>
Sent: Friday, March 30, 2018 9:58:05 AM
To: dev@openwhisk.apache.org
Subject: Re: Invoker HA on Mesos


Tyson Norris <tnorris@adobe.com.INVALID> wrote on 03/27/2018 06:25:59 PM:
>
> Do you have an example of the labels working? I guess the labels are
> changed over time through the lifecycle of the container?
>

Apologies for brutally chopping the email chain; my mail client made a
horrible hash of it.

Right now, all we are doing with Kube labels is to label each action
container with its owning invoker on startup.  This lets us delete orphaned
containers if the invoker crashes and needs to be restarted.  The labeling
happens at [1] and the removal of orphans using the labels at [2].

I think the Kube-native version of part of what you are doing with the
DistributedData for Mesos would be to add and remove additional labels to
give us the option of attaching a new invoker instance to orphaned
containers instead of just destroying them.   Interacting with the
Kubernetes API server to do a labeling operation takes around 10ms, so we
couldn't do this on a truly hot path.  But we could probably afford to
update container labels in parallel with pause/unpause operations, which
could enable re-attachment to any paused containers.

--dave

[1]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-openwhisk%2Fblob%2F0b20df0f725a671f8e51c9e8793116476fd22f76%2Fcore%2Finvoker%2Fsrc%2Fmain%2Fscala%2Fwhisk%2Fcore%2Fcontainerpool%2Fkubernetes%2FKubernetesContainerFactory.scala%23L81&data=02%7C01%7Cddascal%40adobe.com%7Cefb609bc3ac9496ddccf08d596600540%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636580261504215781&sdata=vhkEBD%2BjkTcUDm6csPB5gBg9RZuj%2FViAwGbynBKeE2k%3D&reserved=0
[2]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-openwhisk%2Fblob%2F0b20df0f725a671f8e51c9e8793116476fd22f76%2Fcore%2Finvoker%2Fsrc%2Fmain%2Fscala%2Fwhisk%2Fcore%2Fcontainerpool%2Fkubernetes%2FKubernetesContainerFactory.scala%23L57&data=02%7C01%7Cddascal%40adobe.com%7Cefb609bc3ac9496ddccf08d596600540%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636580261504215781&sdata=njsaRY%2B1DoYpysh7tTMqTaL0lWyPrL8vdnuo%2Bf0sXZY%3D&reserved=0

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message