flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavel Potseluev <potsel...@yandex-team.ru>
Subject Bandwidth throttling of checkpoints uploading to s3
Date Fri, 12 Jul 2019 12:38:59 GMT
<div>Hello!</div><div> </div><div>We use flink with periodically
checkpointing to s3 file system. And when flink uploads checkpoint to s3 it makes high load
to the network. We have found in AWS CLI S3 configuration option called <span style="background:#ffffff;color:#333333;float:none;font:400
14px 'consolas' , 'monaco' , 'menlo' , 'courier new' , monospace;text-align:left;text-decoration-style:initial;text-transform:none;white-space:normal;word-spacing:0px"><a
href="https://docs.aws.amazon.com/cli/latest/topic/s3-config.html#max-bandwidth">max_bandwidth</a>
</span><span style="background:#ffffff;float:none;font-size:14px;font-style:normal;font-weight:400;text-align:left;text-decoration-style:initial;text-transform:none;word-spacing:0px">which
allows to limit rate in bytes per second</span><span style="background:#ffffff;color:#333333;float:none;font:400
14px 'consolas' , 'monaco' , 'menlo' , 'courier new' , monospace;text-align:left;text-decoration-style:initial;text-transform:none;white-space:normal;word-spacing:0px">.</span>
Is there a way to have the same functionality with flink?</div><div> </div><div>-- </div><div
style="background:rgb( 255 , 255 , 255 );color:rgb( 0 , 0 , 0 );font:400 15px 'arial' , sans-serif;text-decoration-style:initial;text-transform:none;white-space:normal;word-spacing:0px">Best
regards,</div><div style="background:rgb( 255 , 255 , 255 );color:rgb( 0 , 0 , 0
);font:400 15px 'arial' , sans-serif;text-decoration-style:initial;text-transform:none;white-space:normal;word-spacing:0px">Pavel
Potseluev</div><div style="background:rgb( 255 , 255 , 255 );color:rgb( 0 , 0 , 0
);font:400 15px 'arial' , sans-serif;text-decoration-style:initial;text-transform:none;white-space:normal;word-spacing:0px">Software
developer, Yandex.Classifieds LLC</div><div> </div>
Mime
View raw message