flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Checkpointing with RocksDB as statebackend
Date Wed, 15 Feb 2017 12:25:01 GMT
What did the More Details link say ?

Thanks 

> On Feb 15, 2017, at 3:11 AM, vinay patil <vinay18.patil@gmail.com> wrote:
> 
> Hi,
> 
> I have kept the checkpointing interval to 6secs and minimum pause between
> checkpoints to 5secs, while testing the pipeline I have observed that that
> for some checkpoints it is taking long time , as you can see in the attached
> snapshot checkpoint id 19 took the maximum time before it gets failed,
> although it has not received any acknowledgements, now during this 10minutes
> the entire pipeline did not make any progress and no data was getting
> processed. (For Ex : In 13minutes 20M records were processed and when the
> checkpoint took time there was no progress for the next 10minutes)
> 
> I have even tried to set max checkpoint timeout to 3min, but in that case as
> well multiple checkpoints were getting failed.
> 
> I have set RocksDB FLASH_SSD_OPTION 
> What could be the issue ? 
> 
> P.S. I am writing to 3 S3 sinks 
> 
> checkpointing_issue.PNG
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/n11640/checkpointing_issue.PNG>
 
> 
> 
> 
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Checkpointing-with-RocksDB-as-statebackend-tp11640.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Mime
View raw message