flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wepngong Benaiah <bwepng...@gmail.com>
Subject Re: Questions on GSoC project: Query optimisation layer for Flink Streaming
Date Fri, 20 Mar 2015 18:08:26 GMT
Hello,


 I am thinking of implementing the following optimization algorithms
for GSOC2015
These are supposed to do statistical graph analysis and streaming
graph optimization
1.  OPERATOR REORDERING
Means changing the order in which the operators appear in the stream
graph to eliminate overheads.
Profitability
Reordering is profitable if it moves selective operators before costly operators
2. REDUNDANCY ELIMINATION
Eliminate redundant computations.
When other operators depend on another, compute that operator once
only and share between other operators.
Profitability
Redundancy elimination is profitable if resources are limited and the
cost of redundant
work is significant.
3. OPERATOR SEPARATION
Separate operators into smaller computational steps.
 Profitability
profitable if it enables other optimizations such as operator reordering.
4. State sharing
Optimize for space by avoiding unnecessary copies of data.
Profitability
Profitable if it reduces stalls due to cache misses or disk
I/O, by decreasing the memory footprint.

My question is, are the above algorithms sufficient , if so, what can
be possible milestones, implementing each algorithm per milestone?

Please i will love to see a sample proposal format or a link to one
Thanks
-- 
Wepngong Ngeh Benaiah

"The similarities of sysadmins and drug dealers: both measure stuff in
Ks, and both have users."

Mime
View raw message