On 15. May 2017, at 17:05, Jim Langston <email@example.com> wrote:Hi all,
I have a long running , streaming app saving checkpoints tothe file system.
What is the layout of the checkpoint directory ? My currentcheckpoint directory has >2000 directories in it , similar to this:
Also, the directory has grown to >3GB
I have a small cluster, and all were started at the same time, nothinghas been restarted, but this is occurring one of the nodes, the others haveabout the same number of directories in the checkpoint directory, butnot nearly as large.
Why are there so many chk-xxxx directories ? And why can they becomeso large ? Is there something I should be setting in the yaml file ?
I was going to just remove them , but it just struck me as odd that thereare so many …