apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Priyanka Gugale <pri...@apache.org>
Subject Re: Apex internal documentation.
Date Tue, 29 Nov 2016 06:10:13 GMT
+1 for documenting this.

I had covered the basic cycle of launch in couple of meetups but there is
no documentation. Also many things like bufferserver management, heartbeat
mechanism, command execution, stats etc are not documented.

-Priyanka

On Tue, Nov 29, 2016 at 11:23 AM, Tushar Gosavi <tushar@apache.org> wrote:

> Do we have documents explaining internals of Apex? Would it be a good
> idea to add document explaining following on apex.apache.org to help
> newcomers understand overall execution flow of application and apex
> capabilities, this might also help them to add new features.
>
> - Startup of application
>   - logical plan formation and properties injection.
>   - copying of jars/resources
>   - handling of old application state.
> - Master initialization
>   - converting logical plan to physical plan
>   - container assignments with locality and affinity
>   - initial checkpoints
>   - deployment
>   - restarting from killed application (-originalId)
> - StramChild functionality
>   - Operator deploy requests and operator lifecycle management.
>   - handling of checkpoints
>   - WindowGenerator
>   - Sending hearbeats
>   - handling of various commands from Stram.
>   - Operator request response model.
>   - Bufferserver management. (purge/reset/disk spooling/back-pressure
> handling)
> - Run-time activities at master.
>   - heartbeat monitoring
>   - stats listeners
>   - recovery from operator failure
>   - dynamical plan change
>
> - Tushar.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message