openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Sicker <boa...@gmail.com>
Subject Re: New architecture proposal
Date Thu, 04 Apr 2019 15:39:51 GMT
Would it make sense to define an OpenWhisk Improvement/Enhancement
Propoposal or similar that various other Apache projects do? We could
call them WHIPs or something. :)

On Thu, 4 Apr 2019 at 09:09, David P Grove <groved@us.ibm.com> wrote:
>
>
> Dominic Kim <style9595@gmail.com> wrote on 04/04/2019 04:37:19 AM:
> >
> > I have proposed a new architecture.
> > https://cwiki.apache.org/confluence/display/OPENWHISK/New+architecture
> +proposal
> >
> > It includes many controversial agendas and breaking changes.
> > So I would like to form a general consensus on them.
> >
>
> Hi Dominic,
>
>         There's much to like about the proposal.  Thank you for writing it
> up.
>
>         One meta-comment is that the work will have to be done in a way so
> there are no actual "breaking changes".  It has to be possible to continue
> to configure the system using the existing architectures while this work
> proceeds.  I would expect this could be done via a new LoadBalancer and
> some deployment options (similar to how Lean OpenWhisk was done).  If work
> needs to be done to generalize the LoadBalancer SPI, that could be done
> early in the process.
>
>         On the proposal itself, I wonder if the complexity of Leader/Follower
> is actually needed?  If a Scheduler crashes, it could be restarted and then
> resume handling its assigned load.  I think there should be enough
> information in etcd for it to recover its current set of assigned
> ContainerProxys and carry on.   Activations in its in memory queues would
> be lost (bigger blast radius than the current architecture), but I don't
> see that the Leader/Follower changes that (seems way too expensive to be
> replicating every activation in the Follower Queues).   The Leader/Follower
> would allow for shorter downtime for those actions assigned to the downed
> Scheduler, but at the cost of significant complexity.  Is it worth it?
>
>         Perhaps related to the Leader/Follower, its not clear to me how
> activation messages are being pulled from the action topic in Kafka during
> the Queue creation window. I think they have to go somewhere (because the
> is a mix of actions on a single Kafka topic and we can't stall other
> actions while waiting for a Queue to be created for a new action), but if
> you don't know yet which Scheduler is going to win the race to be a Leader
> how do you know where to put them?
>
> --dave



-- 
Matt Sicker <boards@gmail.com>

Mime
View raw message